solr ERROR bytes can be at most 32766 in length;

野性酷女 2022-05-16 06:51 221阅读 0赞

solr导入数据的时候报错

Remote error message: Exception writing document id 20170618170826458796 to the index; possible analysis error: Document contains at least one immense term in field=”kng\_point\_content\_JP” (whose UTF8 encoding is longer than the max length 32766), all of which were skipped. Please correct the analyzer to not produce such terms. The prefix of the first immense term is: ‘\[-24, -127, -108, -27, -112, -120, -28, -68, -102, -27, -111, -104, -27, -115, -77, -27, -110, -86, -27, -110, -107, -26, -107, -80, -27, -86, -110, -28, -72, -114\]…’, original message: bytes can be at most 32766 in length; got 36119. Perhaps the document has an indexed string field (solr.StrField) which is too large  
at org.apache.solr.update.processor.DistributedUpdateProcessor.doFinish(DistributedUpdateProcessor.java:972)  
at org.apache.solr.update.processor.DistributedUpdateProcessor.finish(DistributedUpdateProcessor.java:1911)  
at org.apache.solr.update.processor.LogUpdateProcessorFactoryLogUpdateProcessor.finish(LogUpdateProcessorFactory.java:182)atorg.apache.solr.handler.dataimport.SolrWriter.close(SolrWriter.java:61)atorg.apache.solr.handler.dataimport.DocBuilder.execute(DocBuilder.java:274)atorg.apache.solr.handler.dataimport.DataImporter.doFullImport(DataImporter.java:415)atorg.apache.solr.handler.dataimport.DataImporter.runCmd(DataImporter.java:474)atorg.apache.solr.handler.dataimport.DataImporter.lambda  L o g U p d a t e P r o c e s s o r . f i n i s h ( L o g U p d a t e P r o c e s s o r F a c t o r y . j a v a : 182 ) a t o r g . a p a c h e . s o l r . h a n d l e r . d a t a i m p o r t . S o l r W r i t e r . c l o s e ( S o l r W r i t e r . j a v a : 61 ) a t o r g . a p a c h e . s o l r . h a n d l e r . d a t a i m p o r t . D o c B u i l d e r . e x e c u t e ( D o c B u i l d e r . j a v a : 274 ) a t o r g . a p a c h e . s o l r . h a n d l e r . d a t a i m p o r t . D a t a I m p o r t e r . d o F u l l I m p o r t ( D a t a I m p o r t e r . j a v a : 415 ) a t o r g . a p a c h e . s o l r . h a n d l e r . d a t a i m p o r t . D a t a I m p o r t e r . r u n C m d ( D a t a I m p o r t e r . j a v a : 474 ) a t o r g . a p a c h e . s o l r . h a n d l e r . d a t a i m p o r t . D a t a I m p o r t e r . l a m b d a runAsync$0(DataImporter.java:457)  
at java.lang.Thread.run(Thread.java:745)

kng\_point\_content\_JP字段的数据长度大于32766了，

update 表名 set is\_deleted=1 where length(kng\_point\_content) >32700  
我们kng\_point\_content\_JP数据长度大于32700的全部标识为删除，当然提前做好备份，然后重新入solr，  
这里有人可能就问了，为什么不用char\_length,因为kng\_point\_content\_JP字段数据里面包含的有中文，中文是占3个字符长度的，