资讯详情

elasticsearch学习笔记(持续更新)

1.创建网络

docker network create es-net

2.部署es 我在这里使用阿里云服务器，配置需要更高，至少4g内存

docker run -d \  --name es \     -e "ES_JAVA_OPTS=-Xms512m -Xmx512m" \     -e "discovery.type=single-node" \     -e "http.host=0.0.0.0" \     -v es-data:/usr/share/elasticsearch/data \     -v es-plugins:/usr/share/elasticsearch/plugins \     --privileged \     --network es-net \     -p 9200:9200 \     -p 9300:9300 \ elasticsearch:7.12.1

命令解释：

  • -e "cluster.name=es-docker-cluster"：设置集群名称
  • -e "http.host=0.0.0.0"：监控地址可外网访问
  • -e "ES_JAVA_OPTS=-Xms512m -Xmx512m"：内存大小
  • -e "discovery.type=single-node"：非集群模式
  • -v es-data:/usr/share/elasticsearch/data：挂逻辑卷，绑定es的数据目录
  • -v es-logs:/usr/share/elasticsearch/logs：挂逻辑卷，绑定es的日志目录
  • -v es-plugins:/usr/share/elasticsearch/plugins：挂逻辑卷，绑定es的插件目录
  • --privileged：给予逻辑卷访问权
  • --network es-net ：加一个名字es-net的网络中
  • -p 9200:9200：端口映射配置

(可以通过kibana操作elasticsearch)

docker run -d \ --name kibana \ -e ELASTICSEARCH_HOSTS=http://es:9200 \ --network=es-net \ -p 5601:5601 \ kibana:7.12.1

1.通过以下命令

docker volume inspect es-plugins

查看elasticsearch的plugins目录位置Mountpoint

[     { 
                "CreatedAt": "2022-05-06T10:06:34 08:00",         "Driver": "local",         "Labels": null,         "Mountpoint": "/var/lib/docker/volumes/es-plugins/_data",         "Name": "es-plugins",         "Options": null,         "Scope": "local"     } ]

2.把解压过的IK分词器上传到Mountpoint指定的/var/lib/docker/volumes/es-plugins/_data目录 3.重启es

docker restart es

–RestClient 1.引入es的RestHighLevelClient依赖

<dependency>
    <groupId>org.elasticsearch.client</groupId>
    <artifactId>elasticsearch-rest-high-level-client</artifactId>
</dependency>

2.因为SpringBoot默认的ES版本是7.6.2，所以我们需要覆盖默认的ES版本：

<properties>
    <java.version>1.8</java.version>
    <elasticsearch.version>7.12.1</elasticsearch.version>
</properties>

3.初始化RestHighLevelClient：

初始化的代码如下：

RestHighLevelClient client = new RestHighLevelClient(RestClient.builder(
        HttpHost.create("http://192.168.150.101:9200")
));

释放连接

client.close();

mapping是对索引库中文档的约束，常见的mapping属性包括：

  • type：字段数据类型，常见的简单类型有：
    • 字符串：text（可分词的文本）、keyword（精确值，例如：品牌、国家、ip地址）
    • 数值：long、integer、short、byte、double、float、
    • 布尔：boolean
    • 日期：date
    • 对象：object
  • index：是否创建索引，默认为true
  • analyzer：使用哪种分词器
  • properties：该字段的子字段

1.创建索引库的映射

PUT /索引库名称
{ 
       
  "mappings": { 
       
    "properties": { 
       
      "字段名":{ 
       
        "type": "text",
        "analyzer": "ik_smart"
      },
      "字段名2":{ 
       
        "type": "keyword",
        "index": "false"
      },
      "字段名3":{ 
       
        "properties": { 
       
          "子字段": { 
       
            "type": "keyword"
          }
        }
      },
      // ...略
    }
  }
}

需要注意的是如果是坐标的话，在mysql中是存储两个字段，一个经度，一个维度，而在ES中则使用一个字段表示，映射中type为geo_point，以后插入数据的值为 “经度,维度” 在这里插入图片描述

示例：

PUT /heima
{
  "mappings": {
    "properties": {
      "info":{
        "type": "text",
        "analyzer": "ik_smart",
        "copy_to": "all"
      },
      "email":{
        "type": "keyword",
        "index": "falsae"
      },
      "name":{
        "properties": {
          "firstName": {
            "type": "keyword",
       	    "copy_to": "all"
          }
        }
      },
      "location":{
        "type": "geo_point"
      },
      // ... 略
      // ... 略
      // ... 略
      "all":{
        "type": "text",
        "analyzer": "ik_max_word"
      }
    }
  }
}

上面的示例里有一个字段all，其他字段可以通过 “copy_to”: “all”，使得all成为联合索引

package cn.itcast.hotel.constants;

public class HotelConstants { 
       
    public static final String MAPPING_TEMPLATE = "{\n" +
            " \"mappings\": {\n" +
            " \"properties\": {\n" +
            " \"id\": {\n" +
            " \"type\": \"keyword\"\n" +
            " },\n" +
            " \"name\":{\n" +
            " \"type\": \"text\",\n" +
            " \"analyzer\": \"ik_max_word\",\n" +
            " \"copy_to\": \"all\"\n" +
            " },\n" +
            " \"address\":{\n" +
            " \"type\": \"keyword\",\n" +
            " \"index\": false\n" +
            " },\n" +
            " \"price\":{\n" +
            " \"type\": \"integer\"\n" +
            " },\n" +
            " \"score\":{\n" +
            " \"type\": \"integer\"\n" +
            " },\n" +
            " \"brand\":{\n" +
            " \"type\": \"keyword\",\n" +
            " \"copy_to\": \"all\"\n" +
            " },\n" +
            " \"city\":{\n" +
            " \"type\": \"keyword\",\n" +
            " \"copy_to\": \"all\"\n" +
            " },\n" +
            " \"starName\":{\n" +
            " \"type\": \"keyword\"\n" +
            " },\n" +
            " \"business\":{\n" +
            " \"type\": \"keyword\"\n" +
            " },\n" +
            " \"location\":{\n" +
            " \"type\": \"geo_point\"\n" +
            " },\n" +
            " \"pic\":{\n" +
            " \"type\": \"keyword\",\n" +
            " \"index\": false\n" +
            " },\n" +
            " \"all\":{\n" +
            " \"type\": \"text\",\n" +
            " \"analyzer\": \"ik_max_word\"\n" +
            " }\n" +
            " }\n" +
            " }\n" +
            "}";
}

@Test
void createHotelIndex() throws IOException { 
       
    // 1.创建Request对象
    CreateIndexRequest request = new CreateIndexRequest("hotel");
    // 2.准备请求的参数：DSL语句
    request.source(MAPPING_TEMPLATE, XContentType.JSON);
    // 3.发送请求
    client.indices().create(request, RequestOptions.DEFAULT);
}

2.查询索引库

GET /索引库名

@Test void testExistsHotelIndex() throws IOException { // 1.创建Request对象 GetIndexRequest request = new GetIndexRequest(“hotel”); // 2.发送请求 boolean exists = client.indices().exists(request, RequestOptions.DEFAULT); // 3.输出 System.err.println(exists ? “索引库已经存在！” : “索引库不存在！”); } 3.修改索引库 ES无法修改mapping,但是却允许添加新的字段到mapping中，因为不会对倒排索引产生影响

PUT /索引库名/_mapping
{ 
       
  "properties": { 
       
    "新字段名":{ 
       
      "type": "integer"
    }
  }
}

4.删除索引库

DELETE /索引库名


@Test
void testDeleteHotelIndex() throws IOException { 
       
    // 1.创建Request对象
    DeleteIndexRequest request = new DeleteIndexRequest("hotel");
    // 2.发送请求
    client.indices().delete(request, RequestOptions.DEFAULT);
}

1.新增文档

POST /索引库名/_doc/文档id
{ 
       
    "字段1": "值1",
    "字段2": "值2",
    "字段3": { 
       
        "子属性1": "值3",
        "子属性2": "值4"
    },
    // ...
}


POST /heima/_doc/1
{ 
       
    "info": "黑马程序员Java讲师",
    "email": "zy@itcast.cn",
    "name": { 
       
        "firstName": "云",
        "lastName": "赵"
    }
}


@Test
void testAddDocument() throws IOException { 
       
    // 1.根据id查询酒店数据
    Hotel hotel = hotelService.getById(61083L);
    // 2.转换为文档类型
    HotelDoc hotelDoc = new HotelDoc(hotel);
    // 3.将HotelDoc转json
    String json = JSON.toJSONString(hotelDoc);

    // 1.准备Request对象
    IndexRequest request = new IndexRequest("hotel").id(hotelDoc.getId().toString());
    // 2.准备Json文档
    request.source(json, XContentType.JSON);
    // 3.发送请求
    client.index(request, RequestOptions.DEFAULT);
}

2.查询文档

GET /{ 
       索引库名称}/_doc/{ 
       id}


@Test
void testGetDocumentById() throws IOException { 
       
    // 1.准备Request
    GetRequest request = new GetRequest("hotel", "61082");
    // 2.发送请求，得到响应
    GetResponse response = client.get(request, RequestOptions.DEFAULT);
    // 3.解析响应结果
    String json = response.getSourceAsString();

    HotelDoc hotelDoc = JSON.parseObject(json, HotelDoc.class);
    System.out.println(hotelDoc);
}

3.删除文档

DELETE /{ 
       索引库名}/_doc/id值


@Test
void testDeleteDocument() throws IOException { 
       
    // 1.准备Request
    DeleteRequest request = new DeleteRequest("hotel", "61083");
    // 2.发送请求
    client.delete(request, RequestOptions.DEFAULT);
}

4.修改文档 4.1全量修改：直接覆盖原来的文档

PUT /{ 
       索引库名}/_doc/文档id
{ 
       
    "字段1": "值1",
    "字段2": "值2",
    // ... 略
}



PUT /heima/_doc/1
{ 
       
    "info": "黑马程序员高级Java讲师",
    "email": "zy@itcast.cn",
    "name": { 
       
        "firstName": "云",
        "lastName": "赵"
    }
}

在RestClient的API中，全量修改与新增的API完全一致，判断依据是ID

4.2增量修改：修改文档中的部分字段

POST /{ 
       索引库名}/_update/文档id
{ 
       
    "doc": { 
       
         "字段名": "新的值",
    }
}


POST /heima/_update/1
{ 
       
  "doc": { 
       
    "email": "ZhaoYun@itcast.cn"
  }
}


@Test
void testUpdateDocument() throws IOException { 
       
    // 1.准备Request
    UpdateRequest request = new UpdateRequest("hotel", "61083");
    // 2.准备请求参数
    request.doc(
        "price", "952",
        "starName", "四钻"
    );
    // 3.发送请求
    client.update(request, RequestOptions.DEFAULT);
}


@Test
void testBulkRequest() throws IOException { 
       
    // 批量查询酒店数据
    List<Hotel> hotels = hotelService.list();

    // 1.创建Request
    BulkRequest request = new BulkRequest();
    // 2.准备参数，添加多个新增的Request
    for (Hotel hotel : hotels) { 
       
        // 2.1.转换为文档类型HotelDoc
        HotelDoc hotelDoc = new HotelDoc(hotel);
        // 2.2.创建新增文档的Request对象
        request.add(new IndexRequest("hotel")
                    .id(hotelDoc.getId().toString())
                    .source(JSON.toJSONString(hotelDoc), XContentType.JSON));
    }
    // 3.发送请求
    client.bulk(request, RequestOptions.DEFAULT);
}

语法

GET /indexName/_search
{ 
       
  "query": { 
       
    "查询类型": { 
       
      "查询条件": "条件值"
    }
  }
}

1.查询所有

// 查询所有
GET /indexName/_search
{ 
       
  "query": { 
       
    "match_all": { 
       
    }
  }
}


@Test
void testMatchAll() throws IOException { 
       
    // 1.准备Request
    SearchRequest request = new SearchRequest("hotel");
    // 2.准备DSL
    request.source()
        .query(QueryBuilders.matchAllQuery());
    // 3.发送请求
    SearchResponse response = client.search(request, RequestOptions.DEFAULT);

    // 4.解析响应
    handleResponse(response);
}

private void handleResponse(SearchResponse response) { 
       
    // 4.解析响应
    SearchHits searchHits = response.getHits();
    // 4.1.获取总条数
    long total = searchHits.getTotalHits().value;
    System.out.println("共搜索到" + total + "条数据");
    // 4.2.文档数组
    SearchHit[] hits = searchHits.getHits();
    // 4.3.遍历
    for (SearchHit hit : hits) { 
       
        // 获取文档source
        String json = hit.getSourceAsString();
        // 反序列化
        HotelDoc hotelDoc = JSON.parseObject(json, HotelDoc.class);
        System.out.println("hotelDoc = " + hotelDoc);
    }
}

2.全文检索查询

全文检索查询会把需要查询的内容先进行分词，之后再进行查询

查询某一字段

GET /indexName/_search
{ 
       
  "query": { 
       
    "match": { 
       
      "字段名": "查询内容"
    }
  }
}

查询某些字段

GET /indexName/_search
{ 
       
  "query": { 
       
    "multi_match": { 
       
      "query": "TEXT",
      "fields": ["FIELD1", " FIELD12"]
    }
  }
}

需要注意的是，查询多个字段效率会比较低，可以用之前的copy to的联合索引"all"

@Test
void testMatch() throws IOException { 
       
    // 1.准备Request
    SearchRequest request = new SearchRequest("hotel");
    // 2.准备DSL
    request.source()
        .query(QueryBuilders.matchQuery("all", "如家"));
    // 3.发送请求
    SearchResponse response = client.search(request, RequestOptions.DEFAULT);
    // 4.解析响应
    handleResponse(response);

}

3.精准查询 精准查询不会把需要查询的内容先进行分词，查询到的内容必须严格匹配

// term查询
GET /indexName/_search
{ 
       
  "query": { 
       
    "term": { 
       
      "FIELD": { 
       
        "字段名": "查询值"
      }
    }
  }
}


@Test
void testMatch() throws IOException { 
       
    // 1.准备Request
    SearchRequest request = new SearchRequest("hotel");
    // 2.准备DSL
    request.source()
        .query(QueryBuilders.termQuery("city", "杭州"));
    // 3.发送请求
    SearchResponse response = client.search(request, RequestOptions.DEFAULT);
    // 4.解析响应
    handleResponse(response);

}

4.范围查询

// range查询
GET /indexName/_search
{ 
       
  "query": { 
       
    "range": { 
       
      "字段名": { 
       
        "gte": 10, // 这里的gte代表大于等于，gt则代表大于
        "lte": 20 // lte代表小于等于，lt则代表小于
      }
    }
  }
}


@Test
void testMatch() throws IOException { 
       
    // 1.准备Request
    SearchRequest request = new SearchRequest("hotel");
    // 2.准备DSL
    request.source()
        .query(QueryBuilders.rangeQuery("price").gte(
        标签： 61082连接器
 
锐单商城拥有海量元器件数据手册、
          IC替代型号，打造
          电子元器件IC百科大全！

锐单商城 - 一站式电子元器件采购平台