经验首页 前端设计 程序设计 Java相关 移动开发 数据库/运维 软件/图像 大数据/云计算 其他经验
当前位置：技术经验 » 程序设计 » Elasticsearch » 查看文章
Elasticsearch搜索功能的实现（五）-- 实战
来源：cnblogs　　作者：gdwkong　　时间：2023/4/19 9:02:06　　对本文有异议

实战环境

elastic search 8.5.0 + kibna 8.5.0 + springboot 3.0.2 + spring data elasticsearch 5.0.2 + jdk 17

一、集成 spring data elasticsearch

1 添加依赖

  1. <dependency>
  2.     <groupId>org.springframework.boot</groupId>
  3.     <artifactId>spring-boot-starter-data-elasticsearch</artifactId>
  4. </dependency>

2 配置es连接

  1. @Configuration
  2. public class ElasticsearchConfig extends ElasticsearchConfiguration {
  4.     @Override
  5.     public ClientConfiguration clientConfiguration() {
  7.             return ClientConfiguration.builder()
  8.                     .connectedTo("127.0.0.1:9200")
  9.                     .withBasicAuth("elastic", "********")
  10.                     .build();
  12.     }
  13. }

3 配置打印DSL语句

  1. # 日志配置
  2. logging:
  3.   level:
  4.     #es日志
  5.     org.springframework.data.elasticsearch.client.WIRE : trace

二、index及mapping 文件编写

  1. @Data
  2. @Document(indexName = "news") //索引名
  3. @Setting(shards = 1,replicas = 0,refreshInterval = "1s") //shards 分片数 replicas 副本数
  4. @Schema(name = "News",description = "新闻对象")
  5. public class News implements Serializable {
  7.     @Id  //索引主键
  8.     @NotBlank(message = "新闻ID不能为空")
  9.     @Schema(type = "integer",description = "新闻ID",example = "1")
  10.     private Integer id;
  12.     @NotBlank(message = "新闻标题不能为空")
  13.     @Schema(type = "String",description = "新闻标题")
  14.     @MultiField(mainField = @Field(type = FieldType.Text, analyzer = "ik_max_word", searchAnalyzer = "ik_smart"),
  15.             otherFields = {@InnerField(type = FieldType.Keyword, suffix = "keyword") }) //混合类型字段 指定 建立索引时分词器与搜索时入参分词器
  16.     private String title;
  18.     @Schema(type = "LocalDate",description = "发布时间")
  19.     @Field(type = FieldType.Date,format = DateFormat.date)
  20.     private LocalDate pubDate;
  22.     @Schema(type = "String",description = "来源")
  23.     @Field(type = FieldType.Keyword)
  24.     private String source;
  26.     @Schema(type = "String",description = "行业类型代码",example = "1,2,3")
  27.     @Field(type = FieldType.Text,analyzer = "ik_max_word",searchAnalyzer = "ik_smart")
  28.     private String industry;
  30.     @Schema(type = "String",description = "预警类型")
  31.     @Field(type = FieldType.Keyword)
  32.     private String type;
  34.     @Schema(type = "String",description = "涉及公司")
  35.     @Field(type = FieldType.Text, analyzer = "ik_max_word", searchAnalyzer = "ik_smart")
  36.     private String companies;
  38.     @Schema(type = "String",description = "新闻内容")
  39.     @Field(type = FieldType.Text, analyzer = "ik_max_word", searchAnalyzer = "ik_smart")
  40.     private String content;
  42. }

三、DAO层编写

  1. @Repository
  2. public interface NewsRepository extends ElasticsearchRepository<News,Integer> {
  4.     Page<News> findByType(String type, Pageable pageable);
  5. }

四、简单功能实现

4.1 简单功能写法

  1.     /**
  2.      * 新增新闻
  3.      * @param news
  4.      * @return
  5.      */
  6.     @Override
  7.     public void saveNews(News news) {
  8.         newsRepository.save(news);
  9.     }
  11.     /**
  12.      * 删除新闻
  13.      * @param newsId
  14.      */
  15.     @Override
  16.     public void delete(Integer newsId) {
  17.         newsRepository.deleteById(newsId);
  18.     }
  20.     /**
  21.      * 删除新闻索引
  22.      */
  23.     @Override
  24.     public void deleteIndex() {
  25.         operations.indexOps(News.class).delete();
  26.     }
  28.     /**
  29.      * 创建索引
  30.      */
  31.     @Override
  32.     public void createIndex() {
  33.         operations.indexOps(News.class).createWithMapping();
  34.     }
  36.     @Override
  37.     public PageResult findByType(String type) {
  38.         // 先发布日期排序
  39.         Sort sort = Sort.by(new Order(Sort.Direction.DESC, "pubDate"));
  40.         Pageable pageable = PageRequest.of(0,10,sort);
  41.         final Page<News> newsPage = newsRepository.findByType(type, pageable);
  42.         return new PageResult(newsPage.getTotalElements(),newsPage.getContent());
  44.     }

实现效果图片：
image

实际执行的DSL语句：
image

注意： 当指定排序条件时 _score 会被置空

4.2 搜索功能的实现

  1.     @Override
  2.     public PageResult searchNews(NewsPageSearch search) {
  4.         //创建原生查询DSL对象
  5.         final NativeQueryBuilder nativeQueryBuilder = new NativeQueryBuilder();
  7.         // 先发布日期再得分排序
  8.         Sort sort = Sort.by(new Order(Sort.Direction.DESC, "pubDate"),new Order(Sort.Direction.DESC, "_score"));
  10.         Pageable pageable = PageRequest.of(search.getCurPage(), search.getPageSize(),sort);
  13.         final BoolQuery.Builder boolBuilder = new BoolQuery.Builder();
  14.         //过滤条件
  15.         setFilter(search, boolBuilder);
  17.         //关键字搜索
  18.         if (StringUtils.isNotBlank(search.getKeyword())){
  19.             setKeyWordAndHighlightField(search, nativeQueryBuilder, boolBuilder);
  20.         }else {
  21.             nativeQueryBuilder.withQuery(q -> q.bool(boolBuilder.build()));
  22.         }
  24.         nativeQueryBuilder.withPageable(pageable);
  26.         SearchHits<News> searchHits = operations.search(nativeQueryBuilder.build(), News.class);
  27.         //高亮回填封装
  28.         final List<News> newsList = searchHits.getSearchHits().stream()
  29.                 .map(s -> {
  30.                     final News content = s.getContent();
  31.                     final List<String> title = s.getHighlightFields().get("title");
  32.                     final List<String> contentList = s.getHighlightFields().get("content");
  33.                     if (!CollectionUtils.isEmpty(title)){
  34.                         s.getContent().setTitle(title.get(0));
  35.                     }
  36.                     if (!CollectionUtils.isEmpty(contentList)){
  37.                         s.getContent().setContent(contentList.get(0));
  38.                     }
  39.                     return content;
  41.                 }).collect(Collectors.toList());
  43.         return new PageResult<News>(searchHits.getTotalHits(),newsList);
  45.     }
  47.     /**
  48.      * 设置过滤条件 行业类型 来源 预警类型
  49.      * @param search
  50.      * @param boolBuilder
  51.      */
  52.     private void setFilter(NewsPageSearch search, BoolQuery.Builder boolBuilder) {
  53.         //行业类型
  54.         if(StringUtils.isNotBlank(search.getIndustry())){
  55.             // 按逗号拆分
  56.             List<Query> industryQueries = Arrays.asList(search.getIndustry().split(",")).stream().map(p -> {
  57.                 Query.Builder queryBuilder = new Query.Builder();
  58.                 queryBuilder.term(t -> t.field("industry").value(p));
  59.                 return queryBuilder.build();
  60.             }).collect(Collectors.toList());
  61.             boolBuilder.filter(f -> f.bool(t -> t.should(industryQueries)));
  62.         }
  63.         // 来源
  64.         if(StringUtils.isNotBlank(search.getSource())){
  65.             // 按逗号拆分
  66.             List<Query> sourceQueries = Arrays.asList(search.getSource().split(",")).stream().map(p -> {
  67.                 Query.Builder queryBuilder = new Query.Builder();
  68.                 queryBuilder.term(t -> t.field("source").value(p));
  69.                 return queryBuilder.build();
  70.             }).collect(Collectors.toList());
  71.             boolBuilder.filter(f -> f.bool(t -> t.should(sourceQueries)));
  72.         }
  73.         // 预警类型
  74.         if(StringUtils.isNotBlank(search.getType())){
  75.             // 按逗号拆分
  76.             List<Query> typeQueries = Arrays.asList(search.getType().split(",")).stream().map(p -> {
  77.                 Query.Builder queryBuilder = new Query.Builder();
  78.                 queryBuilder.term(t -> t.field("type").value(p));
  79.                 return queryBuilder.build();
  80.             }).collect(Collectors.toList());
  81.             boolBuilder.filter(f -> f.bool(t -> t.should(typeQueries)));
  82.         }
  84.         //范围区间
  85.         if (StringUtils.isNotBlank(search.getStartDate())){
  86.             boolBuilder.filter(f -> f.range(r -> r.field("pubDate")
  87.                     .gte(JsonData.of(search.getStartDate()))
  88.                     .lte(JsonData.of(search.getEndDate()))));
  89.         }
  90.     }
  92.     /**
  93.      * 关键字搜索 title 权重更高
  94.      * 高亮字段  title 、content
  95.      * @param search
  96.      * @param nativeQueryBuilder
  97.      * @param boolBuilder
  98.      */
  99.     private void setKeyWordAndHighlightField(NewsPageSearch search, NativeQueryBuilder nativeQueryBuilder, BoolQuery.Builder boolBuilder) {
  100.         final String keyword = search.getKeyword();
  101.         //查询条件
  102.         boolBuilder.must(b -> b.multiMatch(m -> m.fields("title","content","companies").query(keyword)));
  104.         //高亮
  105.         final HighlightFieldParameters.HighlightFieldParametersBuilder builder = HighlightFieldParameters.builder();
  106.         builder.withPreTags("<font color='red'>")
  107.                 .withPostTags("</font>")
  108.                 .withRequireFieldMatch(true) //匹配才加标签
  109.                 .withNumberOfFragments(0); //显示全文
  110.         final HighlightField titleHighlightField = new HighlightField("title", builder.build());
  111.         final HighlightField contentHighlightField = new HighlightField("content", builder.build());
  112.         final Highlight titleHighlight = new Highlight(List.of(titleHighlightField,contentHighlightField));
  114.         nativeQueryBuilder.withQuery(
  115.                         f -> f.functionScore(
  116.                                 fs -> fs.query(q -> q.bool(boolBuilder.build()))
  117.                                         .functions( FunctionScore.of(func -> func.filter(
  118.                                                         fq -> fq.match(ft -> ft.field("title").query(keyword))).weight(100.0)),
  119.                                                 FunctionScore.of(func -> func.filter(
  120.                                                         fq -> fq.match(ft -> ft.field("content").query(keyword))).weight(20.0)),
  121.                                                 FunctionScore.of(func -> func.filter(
  122.                                                         fq -> fq.match(ft -> ft.field("companies").query(keyword))).weight(10.0)))
  123.                                         .scoreMode(FunctionScoreMode.Sum)
  124.                                         .boostMode(FunctionBoostMode.Sum)
  125.                                         .minScore(1.0)))
  126.                 .withHighlightQuery(new HighlightQuery(titleHighlight,News.class));
  128.     }

实现效果

加权前效果：
image

加权后效果：
image

DSL 语句：

  1. {
  2. 	"from": 0,
  3. 	"size": 6,
  4. 	"sort": [{
  5. 		"pubDate": {
  6. 			"mode": "min",
  7. 			"order": "desc"
  8. 		}
  9. 	}, {
  10. 		"_score": {
  11. 			"order": "desc"
  12. 		}
  13. 	}],
  14. 	"highlight": {
  15. 		"fields": {
  16. 			"title": {
  17. 				"number_of_fragments": 0,
  18. 				"post_tags": ["</font>"],
  19. 				"pre_tags": ["<font color='red'>"]
  20. 			},
  21. 			"content": {
  22. 				"number_of_fragments": 0,
  23. 				"post_tags": ["</font>"],
  24. 				"pre_tags": ["<font color='red'>"]
  25. 			}
  26. 		}
  27. 	},
  28. 	"query": {
  29. 		"function_score": {
  30. 			"boost_mode": "sum",
  31. 			"functions": [{
  32. 				"filter": {
  33. 					"match": {
  34. 						"title": {
  35. 							"query": "立足优势稳住外贸基本盘"
  36. 						}
  37. 					}
  38. 				},
  39. 				"weight": 100.0
  40. 			}, {
  41. 				"filter": {
  42. 					"match": {
  43. 						"content": {
  44. 							"query": "立足优势稳住外贸基本盘"
  45. 						}
  46. 					}
  47. 				},
  48. 				"weight": 20.0
  49. 			}, {
  50. 				"filter": {
  51. 					"match": {
  52. 						"companies": {
  53. 							"query": "立足优势稳住外贸基本盘"
  54. 						}
  55. 					}
  56. 				},
  57. 				"weight": 10.0
  58. 			}],
  59. 			"min_score": 1.0,
  60. 			"query": {
  61. 				"bool": {
  62. 					"filter": [{
  63. 						"bool": {
  64. 							"should": [{
  65. 								"term": {
  66. 									"industry": {
  67. 										"value": "1"
  68. 									}
  69. 								}
  70. 							}, {
  71. 								"term": {
  72. 									"industry": {
  73. 										"value": "2"
  74. 									}
  75. 								}
  76. 							}, {
  77. 								"term": {
  78. 									"industry": {
  79. 										"value": "3"
  80. 									}
  81. 								}
  82. 							}]
  83. 						}
  84. 					}, {
  85. 						"bool": {
  86. 							"should": [{
  87. 								"term": {
  88. 									"source": {
  89. 										"value": "新华社"
  90. 									}
  91. 								}
  92. 							}, {
  93. 								"term": {
  94. 									"source": {
  95. 										"value": "中国经济网"
  96. 									}
  97. 								}
  98. 							}]
  99. 						}
  100. 					}, {
  101. 						"bool": {
  102. 							"should": [{
  103. 								"term": {
  104. 									"type": {
  105. 										"value": "经济简报"
  106. 									}
  107. 								}
  108. 							}, {
  109. 								"term": {
  110. 									"type": {
  111. 										"value": "外贸简报"
  112. 									}
  113. 								}
  114. 							}]
  115. 						}
  116. 					}, {
  117. 						"range": {
  118. 							"pubDate": {
  119. 								"gte": "2023-03-29",
  120. 								"lte": "2023-03-30"
  121. 							}
  122. 						}
  123. 					}],
  124. 					"must": [{
  125. 						"multi_match": {
  126. 							"fields": ["title", "content", "companies"],
  127. 							"query": "立足优势稳住外贸基本盘"
  128. 						}
  129. 					}]
  130. 				}
  131. 			},
  132. 			"score_mode": "sum"
  133. 		}
  134. 	},
  135. 	"track_scores": false,
  136. 	"version": true
  137. }

4.3 接口测试

image

原文链接：https://www.cnblogs.com/gdwkong/p/17331639.html

 友情链接：直通硅谷 　点职佳 　北美留学生论坛

Elasticsearch热门文章

  • ES 14 - (底层原理) Elasticsearch内部如何处理不同type的数据
  • ES 15 - Elasticsearch中的数据类型 (text、keyword、date、geo等)
  • java中elasticsearch7.x关于nested类型的api使用，新增+更新
  • Elasticsearch 映射参数详解 fields
  • [Elasticsearch] ES聚合场景下部分结果数据未返回问题分析
  • ES 22 - Elasticsearch中如何进行日期(数值)范围查询
  • Elasticsearch查询之Term?Query示例解析
  • ElasticSearch 实现分词全文检索 - SpringBoot 完整实现 Demo 附源码【完结篇】
  • ElasticStack学习（十）：深入ElasticSearch搜索之QueryFiltering、多/单字符串的多字段查询
  • ElasticSearch7 设置外网访问

    • Elasticsearch推荐文章

  • 利用Elasticsearch实现地理位置、城市搜索服务
  • 日志采集/分析
  • ELKF（elasticsearch、logstash、kibana、filebeat）搭建及收集nginx日志特昂糖-
  • ElasticSearch性能原理拆解
  • Elasticsearch使用实战以及代码详解
  • 使用 Docker Compose V2 快速搭建日志分析平台 ELK (Elasticsearch、Logstash 和 Kibana)
  • 【解决方案】MySQL5.7 百万数据迁移到 ElasticSearch7.x 的思考
  • K8s部署轻量级日志收集系统EFK（elasticsearch + filebeat + kibana）
  • Docker下elasticsearch8部署、扩容、基本操作实战(含kibana)
  • Elasticsearch 保姆级入门篇

    • 本站QQ群：前端 618073944 | Java 606181507 | Python 626812652 | C/C++ 612253063 | 微信 634508462 | 苹果 692586424 | C#/.net 182808419 | PHP 305140648 | 运维 608723728

    W3xue 的所有内容仅供测试，对任何法律问题及风险不承担任何责任。通过使用本站内容随之而来的风险与本站无关。
    关于我们  |  意见建议  |  捐助我们  |  报错有奖  |  广告合作、友情链接（目前9元/月）请联系QQ：27243702 沸活量
    皖ICP备17017327号-2　皖公网安备34020702000426号