-
ES索引修改mappings与重建reindex详解之修改字段类型
概要
elasticsearch一直在使用，这里总结一下mappings的修改方法，分为两种情况：
- 增加新的字段，这种很简单；
- 修改已有的字段类型，这种就比较麻烦了，需要reindex,对索引进行迁移重建。
一、创建索引
curl -XPUT 'http://127.0.0.1:9200/test?pretty' -H 'Content-Type: application/json' -d '{"settings":{},"mappings":{}}'- 1
{ "settings": { "index": { "number_of_shards": 2, "number_of_replicas": 3, "analysis": { "analyzer": { "char_analyzer": { "filter": ["lowercase"], "type": "custom", "tokenizer": "char_split" } }, "tokenizer": { "char_split": { "token_chars": ["letter", "digit", "punctuation", "symbol"], "min_gram": "1", "type": "nGram", "max_gram": "1" } } } } }, "mappings": { "doc": { "properties": { "id": { "type": "long" }, "pd_name": { "type": "text", "fields": { "keyword": { "type": "keyword", "ignore_above": 256 } }, "analyzer": "char_analyzer" }, "pd_uname": { "type": "text", "fields": { "keyword": { "type": "keyword", "ignore_above": 256 } } }, "product_id": { "type": "long" } } } } }- 1
- 2
- 3
- 4
- 5
- 6
- 7
- 8
- 9
- 10
- 11
- 12
- 13
- 14
- 15
- 16
- 17
- 18
- 19
- 20
- 21
- 22
- 23
- 24
- 25
- 26
- 27
- 28
- 29
- 30
- 31
- 32
- 33
- 34
- 35
- 36
- 37
- 38
- 39
- 40
- 41
- 42
- 43
- 44
- 45
- 46
- 47
- 48
- 49
- 50
- 51
- 52
- 53
- 54
- 55
- 56
1.1、获取mappings
curl -XGET 'http://127.0.0.1:9200/test/_mappings?pretty' { "test" : { "mappings" : { "doc" : { "properties" : { "id" : { "type" : "long" }, "pd_name" : { "type" : "text", "fields" : { "keyword" : { "type" : "keyword", "ignore_above" : 256 } }, "analyzer" : "char_analyzer" }, "pd_uname" : { "type" : "text", "fields" : { "keyword" : { "type" : "keyword", "ignore_above" : 256 } } }, "product_id" : { "type" : "long" } } } } } }- 1
- 2
- 3
- 4
- 5
- 6
- 7
- 8
- 9
- 10
- 11
- 12
- 13
- 14
- 15
- 16
- 17
- 18
- 19
- 20
- 21
- 22
- 23
- 24
- 25
- 26
- 27
- 28
- 29
- 30
- 31
- 32
- 33
- 34
- 35
- 36
二、新增字段修改mappings
增加一个 new_stocks 字段，如下：
curl -XPUT 'http://127.0.0.1:9200/test/doc/_mapping?pretty' -H 'Content-Type: application/json' -d '{"properties":{"new_stocks":{"type":"nested","properties":{"value":{"type":"long"},"conversion":{"type":"long"}}}}}' { "acknowledged" : true }- 1
- 2
- 3
- 4
- 5
再查一下：
curl -XGET 'http://127.0.0.1:9200/test/_mappings?pretty' { "test" : { "mappings" : { "doc" : { "properties" : { "id" : { "type" : "long" }, "new_stocks" : { "type" : "nested", "properties" : { "conversion" : { "type" : "long" }, "value" : { "type" : "long" } } }, "pd_name" : { "type" : "text", "fields" : { "keyword" : { "type" : "keyword", "ignore_above" : 256 } }, "analyzer" : "char_analyzer" }, "pd_uname" : { "type" : "text", "fields" : { "keyword" : { "type" : "keyword", "ignore_above" : 256 } } }, "product_id" : { "type" : "long" } } } } } }- 1
- 2
- 3
- 4
- 5
- 6
- 7
- 8
- 9
- 10
- 11
- 12
- 13
- 14
- 15
- 16
- 17
- 18
- 19
- 20
- 21
- 22
- 23
- 24
- 25
- 26
- 27
- 28
- 29
- 30
- 31
- 32
- 33
- 34
- 35
- 36
- 37
- 38
- 39
- 40
- 41
- 42
- 43
- 44
- 45
- 46
可以看到new_stocks字段已经加上去了。
三、修改OR删除mappings已有字段
如果想把
product_id字段类型由long改成text，并删除id字段呢？此时用第二章节的方案就不行了，需要借用reindex命令重做索引。3.1、创建新索引，将要改字段加进去
curl -XPUT 'http://127.0.0.1:9200/new_test?pretty' -H 'Content-Type: application/json' -d '{"settings":{},"mappings":{}}'- 1
- 2
{ "settings": { "index": { "number_of_shards": 2, "number_of_replicas": 3, "analysis": { "analyzer": { "char_analyzer": { "filter": ["lowercase"], "type": "custom", "tokenizer": "char_split" } }, "tokenizer": { "char_split": { "token_chars": ["letter", "digit", "punctuation", "symbol"], "min_gram": "1", "type": "nGram", "max_gram": "1" } } } } }, "mappings": { "doc": { "properties": { "id": { "type": "long" }, "pd_name": { "type": "text", "fields": { "keyword": { "type": "keyword", "ignore_above": 256 } }, "analyzer": "char_analyzer" }, "pd_uname": { "type": "text", "fields": { "keyword": { "type": "keyword", "ignore_above": 256 } } }, "product_id": { "type": "text", "fields": { "keyword": { "type": "keyword", "ignore_above": 256 } } } } } } }- 1
- 2
- 3
- 4
- 5
- 6
- 7
- 8
- 9
- 10
- 11
- 12
- 13
- 14
- 15
- 16
- 17
- 18
- 19
- 20
- 21
- 22
- 23
- 24
- 25
- 26
- 27
- 28
- 29
- 30
- 31
- 32
- 33
- 34
- 35
- 36
- 37
- 38
- 39
- 40
- 41
- 42
- 43
- 44
- 45
- 46
- 47
- 48
- 49
- 50
- 51
- 52
- 53
- 54
- 55
- 56
- 57
- 58
- 59
- 60
- 61
- 62
3.2、同步数据
curl -XPOST 'http://127.0.0.1:9200/_reindex?pretty' -H 'Content-Type: application/json' -d '{"source":{"index":"test"},"dest":{"index":"new_test"}}'- 1
- 2
3.3、删除原索引并对新索引重命名
#删除 curl -XDELETE 'http://127.0.0.1:9200/test?pretty' #设置别名 curl -XPOST 'http://127.0.0.1:9200/_aliases?pretty' -H 'Content-Type: application/json' -d '{"actions":[{"add":{"index":"new_test","alias":"test"}}]}'- 1
- 2
- 3
- 4
- 5
3.4、同步数据的技巧
- 同步部分字段
#从源头过滤, _source 参数，只同步 id，pd_name两个字段 curl -XPOST 'http://127.0.0.1:9200/_reindex?pretty' -H 'Content-Type: application/json' -d '{"source":{"index":"test","_source":["id","pd_name"]},"dest":{"index":"new_test"}}' #从目的地移除 脚本控制，移除id字段 curl -XPOST 'http://127.0.0.1:9200/_reindex?pretty' -H 'Content-Type: application/json' -d '{"source":{"index":"test"},"dest":{"index":"new_test"},"script":{"lang":"groovy","inline":"ctx._source.remove(\"id\");ctx._source.remove(\"pd_uname\")"}}'- 1
- 2
- 3
- 4
- 5
- 提高同步速率
调整参数size大小，默认1000每批
curl -XPOST 'http://127.0.0.1:9200/_reindex?pretty' -H 'Content-Type: application/json' -d '{"source":{"index":"test","size":5000},"dest":{"index":"new_test"}}'- 1
- 2
四、参考文献
1]:官方文档
2]:ES索引重建reindex详解
3]:ES迁移效率 -
相关阅读:
GIF图像动态生成-JAVA后台生成
K8S深度解析：从入门到精通的全方位指南
中英文说明书丨艾美捷BrdU细胞增殖检测试剂盒
leetcode 13
什么是Nginx？
十一、K8S之持久化存储
生产计划体系完整解决方案(2) : 复杂大规模问题之 - 分区规划
如何批量创建文件夹并命名？
2022年上半年部分团队的总结
C# list泛型集合（线性表）
- 原文地址：https://blog.csdn.net/weixin_38597669/article/details/131757010
