• django+django-haystack+Whoosh(后期切换引擎为Elasticsearch+ik)+Jieba+mysql

    1.前提准备

    环境介绍

    • haystack是django的开源搜索框架，该框架支持Solr, Elasticsearch, Whoosh, *Xapian*搜索引擎，不用更改代码，直接切换引擎，减少代码量。

    • 搜索引擎使用Whoosh，这是一个由纯Python实现的全文搜索引擎，没有二进制文件等，比较小巧，配置比较简单，当然性能自然略低。whoosh和xapian的性能差距还是比较明显。索引和搜索的速度有近4倍的差距，在full cache情况下的性能差距更是达到了60倍。

    • 中文分词+，由于Whoosh自带的是英文分词，对中文的分词支持不是太好，故用jieba替换whoosh的分词组件。

    • Elasticsearch：开源的搜索引擎，本文版本为7.6.0

    • 其他：Python3.6.5, Django2.2 

    安装环境

    1. pip3 install django==2.2 -i https://pypi.douban.com/simple
    2. pip3 install whoosh  -i https://pypi.douban.com/simple
    3. pip3 install django-haystack -i https://pypi.douban.com/simple
    4. pip3 install jieba -i https://pypi.douban.com/simple
    5. pip3 install pymysql -i https://pypi.douban.com/simple
    6. pip3 install elasticsearch==7.6.0 -i https://pypi.douban.com/simple/

    项目结构

    1.  - Project
    2.    - Project
    3.      - settings.py
    4.    - blog
    5.      - models.py

    表结构

    models.py

    1. from django.db import models
    2.  
    3.  
    4. class UserInfo(models.Model):
    5.     username = models.CharField(verbose_name='用户名', max_length=225)
    6.  
    7.     def __str__(self):
    8.         return self.username
    9.  
    10.  
    11. class Tag(models.Model):
    12.     name = models.CharField(verbose_name='标签名称', max_length=225)
    13.  
    14.     def __str__(self):
    15.         return self.name
    16.  
    17.  
    18. class Article(models.Model):
    19.     title = models.CharField(verbose_name='标题', max_length=225)
    20.     content = models.CharField(verbose_name='内容', max_length=225)
    21.     # 外键
    22.     username = models.ForeignKey(verbose_name='用户', to='UserInfo', on_delete=models.DO_NOTHING)
    23.     tag = models.ForeignKey(verbose_name='标签', to='Tag', on_delete=models.DO_NOTHING)
    24.  
    25.     def __str__(self):
    26.         return self.title

    图解

    本文优势

    集全网的django+django-haystack+Whoosh的总结，取其精华，去其糟粕，加入了新的注解。

    如果你想你的es或者Whoosh集成到django上，那你来对地方了

    django+django-haystack+Whoosh+Jieba+mysql

    1. setting.py配置

    1. # 数据库配置
    2. DATABASES = {
    3.     'default': {
    4.         'ENGINE': 'django.db.backends.mysql',
    5.         'NAME': 'dj_ha',
    6.         'USER': 'root',
    7.         'PASSWORD': 'foobared',
    8.         'HOST': '106.14.42.253',
    9.         'PORT': '11111',
    10.     }
    11. }
    12. # app
    13. INSTALLED_APPS = [ 
    14.           'haystack', 
    15. ]
    16. # 本教程使用的是Whoosh，故配置如下
    17. HAYSTACK_CONNECTIONS = {
    18.     'default': {
    19.         'ENGINE': 'haystack.backends.whoosh_backend.WhooshEngine',
    20.         'PATH': os.path.join(os.path.dirname(__file__), 'whoosh_index'),
    21.     },
    22. }
    23. # 自动更新索引
    24. HAYSTACK_SIGNAL_PROCESSOR = 'haystack.signals.RealtimeSignalProcessor'
    25. # 设置每页显示的数目，默认为20，可以自己修改
    26. HAYSTACK_SEARCH_RESULTS_PER_PAGE = 8

    2. 为表模型创建索引，search_indexes.py

    1. 如果你想针对某个app，例如blog做全文检索，则必须在blog的目录下面，建立search_indexes.py文件，文件名不能修改，必须叫search_indexes.py

    1. from haystack import indexes
    2. from .models import Article
    3.  
    4.  
    5. # ArticleIndex：固定写法 表名Index
    6. class ArticleIndex(indexes.SearchIndex, indexes.Indexable):
    7.     # 固定写法  document=True：haystack和搜索引擎，将给text字段分词,建立索引,使用此字段的内容作为索引进行检索
    8.     # use_template=True,使用自己的模板,与document=True进行搭配，自定义检索字段模板(允许谁可以被全文检索,就是谁被建立索引)
    9.     text = indexes.CharField(document=True, use_template=True)
    10.     # 以下字段作为辅助数据,便于调用,最后也不知道怎么辅助,我注释了,也不影响搜索
    11.     # title：写入引擎的字段名,model_attr='title'：相对应的表模型字段名，
    12.     title = indexes.CharField(model_attr='title')
    13.     content = indexes.CharField(model_attr='content')
    14.     username = indexes.CharField(model_attr='username')
    15.     tag = indexes.CharField(model_attr='tag')
    16.  
    17.     def get_model(self):
    18.         # 需要建立索引的模型
    19.         return Article
    20.  
    21.     def index_queryset(self, using=None):
    22.         """Used when the entire index for model is updated."""
    23.         # 写入引擎的数据,必须返回queryset类型
    24.         return self.get_model().objects.all()

    3. 创建被检索的模板(允许谁可以全文检索)

    这个数据模板的作用是对Article.title, Article.content,Article.username.username

    这三个字段建立索引，当检索的时候会对这三个字段的内容，做全文检索匹配。

    数据模板的路径为yourapp/templates/search/indexes/yourapp/note_text.txt，

    例如本例子为blog/templates/search/indexes/blog/article_text.txt  文件名必须为要索引的小写模型类名_text.txt

    1. {{ object.title }}
    2. {{ object.content }}
    3. {{ object.username.username }}

    4. 路由

    urls.py配置(用内置的视图，后期可以自定义，本文也有介绍)

    1. # urls.py
    2. from django.contrib import admin
    3. from django.urls import path, include, re_path
    4.  
    5. urlpatterns = [
    6.     path('admin/', admin.site.urls),
    7.     # 配置的搜索路由，路由可以自定义，include('haystack.urls')固定
    8.     re_path(r'^search/', include('haystack.urls')),
    9. ]

     haystack.urls的内容(内置的，只是我拉出来，让你看一下，不需要进行修改)

    1. from django.urls import path
    2.  
    3. from haystack.views import SearchView
    4.  
    5. urlpatterns = [path("", SearchView(), name="haystack_search")]

    5. search.html

    SearchView()视图函数默认使用的html模板为当前app目录下，

    路径为app名称，/templates/search/search.html

    所以需要在blog/templates/search/下添加search.html文件，内容为

     search.html(原生)

    1. <h2>Searchh2>
    2. <style>
    3.     span.highlighted {
    4.         color: red;
    5.     }
    6. style>
    8. {% load highlight %}
    9.  
    10. <form method="get" action=".">
    11.     <table>
    12.         
    13.         {{ form.as_table }}
    14.  
    15.         {#        {{ form.title.label }}#}
    16.         <tr>
    17.             <td>td>
    18.             <td>
    19.                 <input type="submit" value="Search">
    20.             td>
    21.         tr>
    22.     table>
    23.  
    24.     {% if query %}
    25.         <h3>返回结果h3>
    26.  
    27.         {% for result in page.object_list %}
    28.             
    29.             
    30.             <p>
    31.                 {#                <a href="{{ result.object.get_absolute_url }}">{{ result.object.title }}a>#}
    32.                 <a href="{{ result.object.get_absolute_url }}">{% highlight result.object.title with query %}a>
    33.             p>
    34.             <span>
    35.                 {% highlight result.object.content with query %}
    36.                 {#                {{ result.object.content }}#}
    37.             span>
    38.  
    39.         {% empty %}
    40.             <p>没有查询到结果！！！p>
    41.         {% endfor %}
    42.  
    43.  
    44.         
    45.         {% if page.has_previous or page.has_next %}
    46.             <div>
    47.                 {% if page.has_previous %}<a href="?q={{ query }}&page={{ page.previous_page_number }}">{% endif %}«
    48.                 Previous{% if page.has_previous %}a>{% endif %}
    49.                 |
    50.                 {% if page.has_next %}<a href="?q={{ query }}&page={{ page.next_page_number }}">{% endif %}Next »
    51.                 {% if page.has_next %}a>{% endif %}
    52.             div>
    53.         {% endif %}
    54.     {% else %}
    55.         {# Show some example queries to run, maybe query syntax, something else? #}
    56.     {% endif %}
    57. form>

     后端返回数据介绍

    1. # print(context)
    2.         """
    3.         {
    4.         'query': '刘',
    5.         'form': <ModelSearchForm bound=True, valid=True, fields=(q;models)>,
    6.         'page': <Page 1 of 1>,
    7.         'paginator': <django.core.paginator.Paginator object at 0x0000017D7E0F3470>,
    8.         'suggestion': None}
    9.         """
    10. # print(context.get('page').__dict__)
    11.         """
    12.         {
    13.         'object_list': 
    14.             [
    15.             <SearchResult: blog.article (pk=6)>, 
    16.             <SearchResult: blog.article (pk=8)>, 
    17.             <SearchResult: blog.article (pk=1)>
    18.         ], 
    19.         'number': 1, 
    20.         'paginator': <django.core.paginator.Paginator object at 0x00000257C11A65C0>
    21.         }
    22.         """

    前端返回数据介绍

    1. {% load highlight %}：高亮加载 内置的会省略搜到的内容，之前的内容
    2. {% load my_filters_and_tags %}：自定义高亮
    3. form.as_table：生成表格,里边会自动成成input标签
    4. query：查询的参数
    5. page.object_list：返回的查询一页数据
    6. result：数据对象集
    7. result.object：当前查询的数据对象
    8. page.has_previous or page.has_next：分页

    6. 高亮配置 

    1. # 7.高亮加载
    2. <style>
    3.     span.highlighted {
    4.         color: red;
    5.     }
    6. style>
    7.             
    8. # 1.使用默认值
    9. {% highlight result.summary with query %}
    10. # 案例
    11. <a href="{{ result.object.get_absolute_url }}">
    12. {% highlight result.object.title with query %}
    13. a>
    14.             
    15. # 2.这里我们为 {{ result.summary }}里所有的 {{ query }} 指定了一个<div>div>标签，并且将class设置为highlight_me_please，这样就可以自己通过CSS为{{ query }}添加高亮效果了，怎么样，是不是很科学呢
    16. {% highlight result.summary with query html_tag "div" css_class "highlight_me_please" %}
    17.  
    18. # 3.这里可以限制最终{{ result.summary }}被高亮处理后的长度
    19. {% highlight result.summary with query max_length 40 %}           
    20.  
    21. # 5.自定义使用(后面会介绍)
    22. # 5.4格式
    23. {% myhighlight <text_block> with <query> [css_class "class_name"] [html_tag "span"] [max_length 200] [start_head True] %}
    24. # 5.2使用一
    25. {% myhighlight result.object.content with query css_class "highlighted" html_tag "span" max_length 200 start_head True %}
    26. # 5.3自定义二
    27. {% myhighlight result.object.content with query css_class "highlighted" start_head True %}

    7.自定义

    自定义返回内容

    在app下新建一个文件名称search_views

    1. # 重写SearchView，实现自定义内容
    2. # blog/search_views.py
    3. from haystack.views import SearchView
    4. # 导入模块
    5. from .models import *
    6.  
    7.  
    8. class MySeachView(SearchView):
    9.     def extra_context(self):  # 重载extra_context来添加额外的context内容
    10.         context = super(MySeachView, self).extra_context()
    11.         my_str = '111'
    12.         context['my_str'] = my_str
    13.         # print(context)
    14.         return context

    修改路由

    1. from django.contrib import admin
    2. from django.urls import path, include, re_path
    3. from blog import search_views
    4.  
    5. urlpatterns = [
    6.     path('admin/', admin.site.urls),
    7.     # 原生的
    8.     # re_path(r'^search/', include('haystack.urls')),
    9.     # 自己的
    10.     re_path(r'^search/', search_views.MySeachView(), name='haystack_search'),
    11. ]

    前端使用 

    1. <div>
    2. 圆明园:{{ my_str }}
    3. div>

    自定义search.html模板 

    1. 保证有一个from，get请求，input标签的name=q，value=Search，

    1. <form method="get" action=".">
    2.     <table>
    3. <tr>
    4. <th>
    5. <label for="id_q">Search:label>
    6. th>
    7. <td>
    8. <input type="search" name="q" value="不得不说" id="id_q">
    9. td>
    10. tr>
    11.         <tr>
    12.             <td>
    13.                 <input type="submit" value="Search">
    14.             td>
    15.         tr>
    16.     table>
    17. form>

    自定义高亮显示(原生的会省略)

    新建文件夹templatetags

    添加blog/templatetags/my_filters_and_tags.py 文件和 blog/templatetags/highlighting.py 文件，

    内容如下（源码分别位于haystack/templatetags/lighlight.py 和 haystack/utils/lighlighting.py 中）：
    my_filters_and_tags.py

    1. # encoding: utf-8
    2. from __future__ import absolute_import, division, print_function, unicode_literals
    3.  
    4. from django import template
    5. from django.conf import settings
    6. from django.core.exceptions import ImproperlyConfigured
    7. from django.utils import six
    8.  
    9. from haystack.utils import importlib
    10.  
    11. register = template.Library()
    12.  
    13. class HighlightNode(template.Node):
    14.     def __init__(self, text_block, query, html_tag=None, css_class=None, max_length=None, start_head=None):
    15.         self.text_block = template.Variable(text_block)
    16.         self.query = template.Variable(query)
    17.         self.html_tag = html_tag
    18.         self.css_class = css_class
    19.         self.max_length = max_length
    20.         self.start_head = start_head
    21.  
    22.         if html_tag is not None:
    23.             self.html_tag = template.Variable(html_tag)
    24.  
    25.         if css_class is not None:
    26.             self.css_class = template.Variable(css_class)
    27.  
    28.         if max_length is not None:
    29.             self.max_length = template.Variable(max_length)
    30.  
    31.         if start_head is not None:
    32.             self.start_head = template.Variable(start_head)
    33.  
    34.     def render(self, context):
    35.         text_block = self.text_block.resolve(context)
    36.         query = self.query.resolve(context)
    37.         kwargs = {}
    38.  
    39.         if self.html_tag is not None:
    40.             kwargs['html_tag'] = self.html_tag.resolve(context)
    41.  
    42.         if self.css_class is not None:
    43.             kwargs['css_class'] = self.css_class.resolve(context)
    44.  
    45.         if self.max_length is not None:
    46.             kwargs['max_length'] = self.max_length.resolve(context)
    47.  
    48.         if self.start_head is not None:
    49.             kwargs['start_head'] = self.start_head.resolve(context)
    50.  
    51.         # Handle a user-defined highlighting function.
    52.         if hasattr(settings, 'HAYSTACK_CUSTOM_HIGHLIGHTER') and settings.HAYSTACK_CUSTOM_HIGHLIGHTER:
    53.             # Do the import dance.
    54.             try:
    55.                 path_bits = settings.HAYSTACK_CUSTOM_HIGHLIGHTER.split('.')
    56.                 highlighter_path, highlighter_classname = '.'.join(path_bits[:-1]), path_bits[-1]
    57.                 highlighter_module = importlib.import_module(highlighter_path)
    58.                 highlighter_class = getattr(highlighter_module, highlighter_classname)
    59.             except (ImportError, AttributeError) as e:
    60.                 raise ImproperlyConfigured("The highlighter '%s' could not be imported: %s" % (settings.HAYSTACK_CUSTOM_HIGHLIGHTER, e))
    61.         else:
    62.             from .highlighting import Highlighter
    63.             highlighter_class = Highlighter
    64.  
    65.         highlighter = highlighter_class(query, **kwargs)
    66.         highlighted_text = highlighter.highlight(text_block)
    67.         return highlighted_text
    68.  
    69.  
    70. @register.tag
    71. def myhighlight(parser, token):
    72.     """
    73.     Takes a block of text and highlights words from a provided query within that
    74.     block of text. Optionally accepts arguments to provide the HTML tag to wrap
    75.     highlighted word in, a CSS class to use with the tag and a maximum length of
    76.     the blurb in characters.
    77.     Syntax::
    78.         {% highlight  with  [css_class "class_name"] [html_tag "span"] [max_length 200] %}
    79.     Example::
    80.         # Highlight summary with default behavior.
    81.         {% highlight result.summary with request.query %}
    82.         # Highlight summary but wrap highlighted words with a div and the
    83.         # following CSS class.
    84.         {% highlight result.summary with request.query html_tag "div" css_class "highlight_me_please" %}
    85.         # Highlight summary but only show 40 characters.
    86.         {% highlight result.summary with request.query max_length 40 %}
    87.     """
    88.     bits = token.split_contents()
    89.     tag_name = bits[0]
    90.  
    91.     if not len(bits) % 2 == 0:
    92.         raise template.TemplateSyntaxError(u"'%s' tag requires valid pairings arguments." % tag_name)
    93.  
    94.     text_block = bits[1]
    95.  
    96.     if len(bits) < 4:
    97.         raise template.TemplateSyntaxError(u"'%s' tag requires an object and a query provided by 'with'." % tag_name)
    98.  
    99.     if bits[2] != 'with':
    100.         raise template.TemplateSyntaxError(u"'%s' tag's second argument should be 'with'." % tag_name)
    101.  
    102.     query = bits[3]
    103.  
    104.     arg_bits = iter(bits[4:])
    105.     kwargs = {}
    106.  
    107.     for bit in arg_bits:
    108.         if bit == 'css_class':
    109.             kwargs['css_class'] = six.next(arg_bits)
    110.  
    111.         if bit == 'html_tag':
    112.             kwargs['html_tag'] = six.next(arg_bits)
    113.  
    114.         if bit == 'max_length':
    115.             kwargs['max_length'] = six.next(arg_bits)
    116.  
    117.         if bit == 'start_head':
    118.             kwargs['start_head'] = six.next(arg_bits)
    119.  
    120.     return HighlightNode(text_block, query, **kwargs)

    highlighting.py

    1. # encoding: utf-8
    2.  
    3. from __future__ import absolute_import, division, print_function, unicode_literals
    4.  
    5. from django.utils.html import strip_tags
    6.  
    7.  
    8. class Highlighter(object):
    9.     #默认值
    10.     css_class = 'highlighted'
    11.     html_tag = 'span'
    12.     max_length = 200
    13.     start_head = False
    14.     text_block = ''
    15.  
    16.     def __init__(self, query, **kwargs):
    17.         self.query = query
    18.  
    19.         if 'max_length' in kwargs:
    20.             self.max_length = int(kwargs['max_length'])
    21.  
    22.         if 'html_tag' in kwargs:
    23.             self.html_tag = kwargs['html_tag']
    24.  
    25.         if 'css_class' in kwargs:
    26.             self.css_class = kwargs['css_class']
    27.  
    28.         if 'start_head' in kwargs:
    29.             self.start_head = kwargs['start_head']
    30.  
    31.         self.query_words = set([word.lower() for word in self.query.split() if not word.startswith('-')])
    32.  
    33.     def highlight(self, text_block):
    34.         self.text_block = strip_tags(text_block)
    35.         highlight_locations = self.find_highlightable_words()
    36.         start_offset, end_offset = self.find_window(highlight_locations)
    37.         return self.render_html(highlight_locations, start_offset, end_offset)
    38.  
    39.     def find_highlightable_words(self):
    40.         # Use a set so we only do this once per unique word.
    41.         word_positions = {}
    42.  
    43.         # Pre-compute the length.
    44.         end_offset = len(self.text_block)
    45.         lower_text_block = self.text_block.lower()
    46.  
    47.         for word in self.query_words:
    48.             if not word in word_positions:
    49.                 word_positions[word] = []
    50.  
    51.             start_offset = 0
    52.  
    53.             while start_offset < end_offset:
    54.                 next_offset = lower_text_block.find(word, start_offset, end_offset)
    55.  
    56.                 # If we get a -1 out of find, it wasn't found. Bomb out and
    57.                 # start the next word.
    58.                 if next_offset == -1:
    59.                     break
    60.  
    61.                 word_positions[word].append(next_offset)
    62.                 start_offset = next_offset + len(word)
    63.  
    64.         return word_positions
    65.  
    66.     def find_window(self, highlight_locations):
    67.         best_start = 0
    68.         best_end = self.max_length
    69.  
    70.         # First, make sure we have words.
    71.         if not len(highlight_locations):
    72.             return (best_start, best_end)
    73.  
    74.         words_found = []
    75.  
    76.         # Next, make sure we found any words at all.
    77.         for word, offset_list in highlight_locations.items():
    78.             if len(offset_list):
    79.                 # Add all of the locations to the list.
    80.                 words_found.extend(offset_list)
    81.  
    82.         if not len(words_found):
    83.             return (best_start, best_end)
    84.  
    85.         if len(words_found) == 1:
    86.             return (words_found[0], words_found[0] + self.max_length)
    87.  
    88.         # Sort the list so it's in ascending order.
    89.         words_found = sorted(words_found)
    90.  
    91.         # We now have a denormalized list of all positions were a word was
    92.         # found. We'll iterate through and find the densest window we can by
    93.         # counting the number of found offsets (-1 to fit in the window).
    94.         highest_density = 0
    95.  
    96.         if words_found[:-1][0] > self.max_length:
    97.             best_start = words_found[:-1][0]
    98.             best_end = best_start + self.max_length
    99.  
    100.         for count, start in enumerate(words_found[:-1]):
    101.             current_density = 1
    102.  
    103.             for end in words_found[count + 1:]:
    104.                 if end - start < self.max_length:
    105.                     current_density += 1
    106.                 else:
    107.                     current_density = 0
    108.  
    109.                 # Only replace if we have a bigger (not equal density) so we
    110.                 # give deference to windows earlier in the document.
    111.                 if current_density > highest_density:
    112.                     best_start = start
    113.                     best_end = start + self.max_length
    114.                     highest_density = current_density
    115.  
    116.         return (best_start, best_end)
    117.  
    118.     def render_html(self, highlight_locations=None, start_offset=None, end_offset=None):
    119.         # Start by chopping the block down to the proper window.
    120.         #text_block为内容，start_offset,end_offset分别为第一个匹配query开始和按长度截断位置
    121.         text = self.text_block[start_offset:end_offset]
    122.  
    123.         # Invert highlight_locations to a location -> term list
    124.         term_list = []
    125.  
    126.         for term, locations in highlight_locations.items():
    127.             term_list += [(loc - start_offset, term) for loc in locations]
    128.  
    129.         loc_to_term = sorted(term_list)
    130.  
    131.         # Prepare the highlight template
    132.         if self.css_class:
    133.             hl_start = '<%s class="%s">' % (self.html_tag, self.css_class)
    134.         else:
    135.             hl_start = '<%s>' % (self.html_tag)
    136.  
    137.         hl_end = '' % self.html_tag
    138.  
    139.         # Copy the part from the start of the string to the first match,
    140.         # and there replace the match with a highlighted version.
    141.         #matched_so_far最终求得为text中最后一个匹配query的结尾
    142.         highlighted_chunk = ""
    143.         matched_so_far = 0
    144.         prev = 0
    145.         prev_str = ""
    146.  
    147.         for cur, cur_str in loc_to_term:
    148.             # This can be in a different case than cur_str
    149.             actual_term = text[cur:cur + len(cur_str)]
    150.  
    151.             # Handle incorrect highlight_locations by first checking for the term
    152.             if actual_term.lower() == cur_str:
    153.                 if cur < prev + len(prev_str):
    154.                     continue
    155.  
    156.                 #分别添上每个query+其后面的一部分（下一个query的前一个位置）
    157.                 highlighted_chunk += text[prev + len(prev_str):cur] + hl_start + actual_term + hl_end
    158.                 prev = cur
    159.                 prev_str = cur_str
    160.  
    161.                 # Keep track of how far we've copied so far, for the last step
    162.                 matched_so_far = cur + len(actual_term)
    163.  
    164.         # Don't forget the chunk after the last term
    165.         #加上最后一个匹配的query后面的部分
    166.         highlighted_chunk += text[matched_so_far:]
    167.  
    168.         #如果不要开头not start_head才加点
    169.         if start_offset > 0 and not self.start_head:
    170.             highlighted_chunk = '...%s' % highlighted_chunk
    171.  
    172.         if end_offset < len(self.text_block):
    173.             highlighted_chunk = '%s...' % highlighted_chunk
    174.  
    175.         #可见到目前为止还不包含start_offset前面的，即第一个匹配的前面的部分（text_block[:start_offset]），如需展示(当start_head为True时)便加上
    176.         if self.start_head:
    177.             highlighted_chunk = self.text_block[:start_offset] + highlighted_chunk
    178.         return highlighted_chunk

    前端使用

    1. <style>
    2.     span.highlighted {
    3.         color: red;
    4.     }
    5. style>
    6.  
    7. {% load my_filters_and_tags %}
    8.  
    9.  
    10.  
    11. {% myhighlight result.object.content with query css_class "highlighted" html_tag "span" max_length 200 start_head True %}

     8. 目前位置搜索已经完成，可以重建索引，同步数据，测试一下

    python manage.py rebuild_index

    9.jieba分词器配置

    9.1 先从python包中复制whoosh_backend.py到app中，并改名为whoosh_cn_backend.py

    文件路径：\site-packages\haystack\backends\whoosh_backend.py

     在这里插入图片描述

    复制到的路径：

    9.2 对whoosh_cn_backend.py做以下修改：

    1. 1、导入 ChineseAnalyze
    2. from jieba.analyse import ChineseAnalyzer
    3.  
    4. 2、替换schema_fields[field_class.index_fieldname] = TEXT(下的analyzer
    5. analyzer=ChineseAnalyzer(),

     9.3 在django的配置文件中，修改搜索引擎

    1. HAYSTACK_CONNECTIONS = {
    2.     'default': {
    3.         # 设置haystack的搜索引擎
    4.         'ENGINE': 'blog.whoosh_cn_backend.WhooshEngine',
    5.         # 'ENGINE': 'haystack.backends.whoosh_backend.WhooshEngine',
    6.         # 设置索引文件的位置
    7.         'PATH': os.path.join(BASE_DIR, 'whoosh_index'),
    8.     }
    9. }

    10 django+django-haystack+Elasticsearch7.5+ik+mysql

    10.0 切换成es引擎，除了settings.py和把jieba换成ik，其他步骤跟上面的都一样

    如果一开始，就是奔着es+ik来的，那步骤9 jieba分词器配置 不用看，直接从步骤8跳到这里来

    10.1 安装es，ik

    基于docker安装Elasticsearch+ElasticSearch-Head+IK分词器_骑台风走的博客-CSDN博客基于docker安装Elasticsearch+ElasticSearch-Head+IK分词器https://blog.csdn.net/qq_52385631/article/details/126567059?spm=1001.2014.3001.5501

    10.2 使用ik重写es7.5引擎

    10.2.1 新建elasticsearch_ik_backend.py(在自己的app下)

    在 blog应用下新建名为 elasticsearch7_ik_backend.py 的文件，继承 Elasticsearch7SearchBackend（后端） 和 Elasticsearch7SearchEngine（搜索引擎） 并重写建立索引时的分词器设置

    1. from haystack.backends.elasticsearch7_backend import Elasticsearch7SearchBackend, Elasticsearch7SearchEngine
    2.  
    3. """
    4. 分析器主要有两种情况会被使用：
    5. 第一种是插入文档时，将text类型的字段做分词然后插入倒排索引，
    6. 第二种就是在查询时，先对要查询的text类型的输入做分词，再去倒排索引搜索
    8. 如果想要让 索引 和 查询 时使用不同的分词器，ElasticSearch也是能支持的，只需要在字段上加上search_analyzer参数
    10. 在索引时，只会去看字段有没有定义analyzer，有定义的话就用定义的，没定义就用ES预设的
    12. 在查询时，会先去看字段有没有定义search_analyzer，如果没有定义，就去看有没有analyzer，再没有定义，才会去使用ES预设的
    13. """
    14.  
    15. DEFAULT_FIELD_MAPPING = {
    16.     "type": "text",
    17.     "analyzer": "ik_max_word",
    18.     # "analyzer": "ik_smart",
    19.     "search_analyzer": "ik_smart"
    20.  
    21. }
    22.  
    23.  
    24. class Elasticsearc7IkSearchBackend(Elasticsearch7SearchBackend):
    25.     def __init__(self, *args, **kwargs):
    26.         self.DEFAULT_SETTINGS['settings']['analysis']['analyzer']['ik_analyzer'] = {
    27.             "type": "custom",
    28.             "tokenizer": "ik_max_word",
    29.             # "tokenizer": "ik_smart",
    30.         }
    31.         super(Elasticsearc7IkSearchBackend, self).__init__(*args, **kwargs)
    32.  
    33.  
    34. class Elasticsearch7IkSearchEngine(Elasticsearch7SearchEngine):
    35.     backend = Elasticsearc7IkSearchBackend

     10.3 修改settings.py(切换成功)

    1. # es 7.x配置
    2. HAYSTACK_CONNECTIONS = {
    3.     'default': {
    4.         # 'ENGINE': 'haystack.backends.elasticsearch7_backend.Elasticsearch7SearchEngine',
    5.         'ENGINE': 'blog.elasticsearch_ik_backend.Elasticsearch7IkSearchEngine',
    6.         # 'URL': 'http://106.14.42.253:9200/',
    7.         'URL': 'http://106.14.42.253:9200/',
    8.         # elasticsearch建立的索引库的名称，一般使用项目名作为索引库
    9.         'INDEX_NAME': 'elastic_new',
    10.     },
    11. }

    10.4 重建索引，同步数据

    python manage.py rebuild_index

    10.5 补充

    10.5.1 未成功切换成ik

    haystack 原先加载的是 ...\venv\Lib\site-packages\haystack\backends 文件夹下的 elasticsearch7_backend.py 文件，打开即可看到 elasticsearch7 引擎的默认配置 

    若用上述方法建立出来的索引字段仍使用 snowball 分词器，则将原先elasticsearch7_backend.py 文件中的 DEFAULT_FIELD_MAPPING 也修改为 ik 分词器（或许是因为版本问题）

    位置：D:\py_virtualenv\dj_ha\Lib\site-packages\haystack\backends\elasticsearch7_backend.py

    修改内容：

    1. DEFAULT_FIELD_MAPPING = {
    2.     "type": "text",
    3.     "analyzer": "ik_max_word",
    4.     "search_analyzer": "ik_smart",
    5. }

    10.5.2 es6版本加入ik，重写引擎

    1. from haystack.backends.elasticsearch_backend import ElasticsearchSearchBackend
    2. from haystack.backends.elasticsearch_backend import ElasticsearchSearchEngine
    3. class IKSearchBackend(ElasticsearchSearchBackend):
    4.     DEFAULT_ANALYZER = "ik_max_word" # 这里将 es 的 默认 analyzer 设置为 ik_max_word
    5.  
    6.     def __init__(self, connection_alias, **connection_options):
    7.         super().__init__(connection_alias, **connection_options)
    8.  
    9.     def build_schema(self, fields):
    10.         content_field_name, mapping = super(IKSearchBackend, self).build_schema(fields)
    11.         for field_name, field_class in fields.items():
    12.             field_mapping = mapping[field_class.index_fieldname]
    13.             if field_mapping["type"] == "string" and field_class.indexed:
    14.                 if not hasattr(
    15.                     field_class, "facet_for"
    16.                 ) and not field_class.field_type in ("ngram", "edge_ngram"):
    17.                     field_mapping["analyzer"] = getattr(
    18.                         field_class, "analyzer", self.DEFAULT_ANALYZER
    19.                     )
    20.             mapping.update({field_class.index_fieldname: field_mapping})
    21.         return content_field_name, mapping
    22.  
    23.  
    24. class IKSearchEngine(ElasticsearchSearchEngine):
    25.     backend = IKSearchBackend

    11.实时更新索原理：采用信号

    配置

    1. # 在django配置文件中，添加索引值，文章更新的时候，就会自动更新索引值
    2. HAYSTACK_SIGNAL_PROCESSOR = 'haystack.signals.RealtimeSignalProcessor'

    RealtimeSignalProcessor源码如下：

    1. class RealtimeSignalProcessor(BaseSignalProcessor):
    2.     """
    3.     Allows for observing when saves/deletes fire & automatically updates the
    4.     search engine appropriately.
    5.     当 检索对象出现保存或者删除的时候更新索引值。
    6.     """
    7.     def setup(self):
    8.         # Naive (listen to all model saves).
    9.         models.signals.post_save.connect(self.handle_save)
    10.         models.signals.post_delete.connect(self.handle_delete)
    11.      
    12.         # Efficient would be going through all backends & collecting all models
    13.         # being used, then hooking up signals only for those.
    14.     def teardown(self):
    15.         # Naive (listen to all model saves).
    16.         models.signals.post_save.disconnect(self.handle_save)
    17.         models.signals.post_delete.disconnect(self.handle_delete)
    18.         # Efficient would be going through all backends & collecting all models
    19.         # being used, then disconnecting signals only for those.

    本文借鉴

    Django haystack实现全文搜索 - -零 - 博客园 (cnblogs.com)
    (9条消息) django-haystack全文检索详细教程_AC_hell的博客-CSDN博客
    (9条消息) Django全文检索Haystack模块_NQ31的博客-CSDN博客_django haystack
    (9条消息) django+drf_haystack+elasticsearch_骑台风走的博客-CSDN博客

    (5条消息) Haystack 使用 Elasticsearch 建立索引时 修改为中文分词器_SevenBerry的博客-CSDN博客_elasticsearch 修改字段分词器

    (5条消息) Elasticsearch中analyzer和search_analyzer的区别_chuixue24的博客-CSDN博客

  • 相关阅读:
    信创优选，国产开源。Solon v2.5.3 发布
    一次搞懂SpringBoot核心原理：自动配置、事件驱动、Condition
    【线性代数基础进阶】向量-part3
    第十七章 Excel操作
    MySQL数据库的MVCC详解
    全世界IT人苦竞业久矣！美国FTC宣布全面废除员工竞业协议
    整合小分子物质KGN的壳聚糖-透明质酸水凝胶/基于琥珀酰壳聚糖-氧化透明质酸的水凝胶的研究制备
    ImmunoChemistry艾美捷平底MicroTiter平板说明书
    L8.linux命令每日一练 -- 第二章 文件和目录操作命令 -- rm和rmdir命令
    阿里云云原生一体化数仓—离线实时一体化新能力解读
  • 原文地址：https://blog.csdn.net/qq_52385631/article/details/126590931