搜索引擎Zend_lucene

怼烎@ 2022-09-30 14:49 140阅读 0赞

Zend Lucene

**1.General**

Zend\_Search\_Lucene is a general purpose text search engine written entirely in PHP 5. it stores its index on the filesystem and does not require a database server.

**2. How to install Zend Lucene**

l DownLoad WebSite : [http://www.zend.com/community/downloads][http_www.zend.com_community_downloads]

l Zend Framework version : Zend Framework 1.9 minimal

Download Zend Framework 1.9 minimal from DownLoad WebSite.

Remove everything from Zend Folder but remain following files and directories:

l Exception.php

l Loader/

l Loader.php

l Search/

**3.How to create an index.**

an example of creating an index as below:

<?php

//File Name: createindex.php

require\_once 'Zend/Search/Lucene.php';

$productsData= array(

0=>array("PID"=>1,"url"=>"http://www.cybozu.jp","productName"=>"garoon","Description"=>"garoon Description","lag"=>"en"),

1=>array("PID"=>2,"url"=>"http://www.cybozu.jp","productName"=>"share360","Description"=>"share360 Description","lag"=>"en"),

2=>array("PID"=>3,"url"=>"http://www.cybozu.jp a","productName"=>"日本語の製品名前","Description"=>"日本語の製品","lag"=>"jp"),

3=>array("PID"=>4,"url"=>"http://www.cybozu.jp a","productName"=>"中文产品名","Description"=>"中文产品描述","lag"=>"zh")

);

$index=new Zend\_Search\_Lucene('index',true);

$doc = new Zend\_Search\_Lucene\_Document();

foreach ($productsData as $productData)

$doc->addField(Zend\_Search\_Lucene\_Field::keyword('PID', $productData\['PID'\], 'UTF-8'));

$doc->addField(Zend\_Search\_Lucene\_Field::Text('url', $productData\['url'\], 'UTF-8'));

$doc->addField(Zend\_Search\_Lucene\_Field::Text('productName', $productData\['productName'\], 'UTF-8'));

$doc->addField(Zend\_Search\_Lucene\_Field::Text('Description', $productData\['Description'\], 'UTF-8'));

$doc->addField(Zend\_Search\_Lucene\_Field::unIndexed('lan', $productData\['lan'\], 'UTF-8'));

$index->addDocument($doc);

$index->commit();

$index->optimize();

echo 'index has been created!';

In KB project, index data is come from database, using method above , We can index all the text from database.

**4.Searching index**

After creating an index , We can search index as below:

<?php

//File Name: search.php

require\_once('Zend/Search/Lucene.php');

$index = new Zend\_Search\_Lucene('index');

$keywords='garoon';

echo "Index contains \{ $index->count()\} documents./n";

$query = Zend\_Search\_Lucene\_Search\_QueryParser::parse( $keywords, 'utf-8' );

$hits = $index->find($query);

foreach ($hits as $hit)

echo 'PID: '.$hit->PID.' ';

echo 'Score: '.$hit->score.' ';

echo 'url: '.$hit->url.' ';

echo 'productName: '.$hit->productName.' ';

echo 'lan: '.$hit->lan.' ';

If we want to search the text for multiple language, We can get value of lan , and then display different results by lan.

**5.delete and update index.**

If we want to update index , first we must find the document in index by keyword, then delete it ,after deleting the old document ,We can add a new document. This is an example to update an index. We delete PID :1 product,and update the description.

<?php

require\_once('Zend/Search/Lucene.php');

$index = new Zend\_Search\_Lucene('index');

//new product data to update

$productNewData =array("PID"=>1,"url"=>"http://www.cybozu.jp","productName"=>"garoon","Description"=>"update garoon Description","lan"=>"en");

$keywords="PID:1";

$hits = $index->find($keywords);

//Delete PID:1

foreach ($hits as $hit)

echo 'PID: '.$hit->PID .'has been deleted ';

$index->delete($hit->id);

$index->commit();

//add new product data to index

$doc = new Zend\_Search\_Lucene\_Document();

$doc->addField(Zend\_Search\_Lucene\_Field::keyword('PID', $productNewData\['PID'\], 'UTF-8'));

$doc->addField(Zend\_Search\_Lucene\_Field::Text('url', $productNewData\['url'\], 'UTF-8'));

$doc->addField(Zend\_Search\_Lucene\_Field::Text('productName', $productNewData\['productName'\], 'UTF-8'));

$doc->addField(Zend\_Search\_Lucene\_Field::Text('Description', $productNewData\['Description'\], 'UTF-8'));

$doc->addField(Zend\_Search\_Lucene\_Field::unIndexed('lan', $productNewData\['lan'\], 'UTF-8'));

$index->addDocument($doc);

$index->commit();

$index->optimize();

**6.How to search japanese or chinese text by lucene.**

As default , lucene can only search English text.But in this project , we must search the text by English, Japanese and Chinese. So we have to change default analyzer of Lucene.

This is an extend of default analyzer of Lucene as below:

<?php

// File Name:chinese.php

require\_once 'Zend/Search/Lucene/Analysis/Analyzer.php';

require\_once 'Zend/Search/Lucene/Analysis/Analyzer/Common.php';

class CN\_Lucene\_Analyzer extends Zend\_Search\_Lucene\_Analysis\_Analyzer\_Common

private $\_position;

private $\_cnStopWords = array( );

public function setCnStopWords( $cnStopWords )

$this->\_cnStopWords = $cnStopWords;

/\*\*

\* Reset token stream

\*/

public function reset()

$this->\_position = 0;

$search = array(",", "/", "//", ".", ";", ":", "/"", "!", "~", "\`", "^", "(", ")", "?", "-", "'", "<", ">", "$", "&","%", "\#", "@", "+", "=", "\{", "\}", "\[", "\]", "：", "）", "（", "．", "。", "，", "！", "；", "“", "”", "‘", "’", "［", "］","、", "—", "　", "《", "》", "－", "…", "【", "】", "？", "￥" );

$this->\_input = str\_replace( $search, '', $this->\_input );

$this->\_input = str\_replace( $this->\_cnStopWords, ' ', $this->\_input );

/\*\*

\* Tokenization stream API

\* Get next token

\* Returns null at the end of stream

\* **@return** Zend\_Search\_Lucene\_Analysis\_Token|null

\*/

public function nextToken()

if ($this->\_input === null)

return null;

$len = strlen($this->\_input);

//print "Old string：".$this->\_input." ";

while ($this->\_position < $len)

// Delete space at the begining

while ($this->\_position < $len &&$this->\_input\[$this->\_position\]==' ' )

$this\->\_position++;

$termStartPosition = $this->\_position;

$temp\_char = $this->\_input\[$this->\_position\];

$isCnWord = false;

if(ord($temp\_char)>127)

$i = 0;

while( $this->\_position < $len && ord( $this->\_input\[$this->\_position\] )>127 )

$this->\_position = $this->\_position + 3;

$i ++;

if($i==2)

$isCnWord = true;

break;

if($i==1) continue;

else

while ($this->\_position < $len && ctype\_alnum( $this->\_input\[$this->\_position\] ))

$this\->\_position++;

//echo $this->\_position.":".$this->\_input\[$this->\_position-1\]."/n";

if ($this->\_position == $termStartPosition)

$this\->\_position++;

continue;

$tmp\_str = substr($this->\_input, $termStartPosition, $this->\_position - $termStartPosition);

$token = new Zend\_Search\_Lucene\_Analysis\_Token( $tmp\_str, $termStartPosition,$this->\_position );

$token = $this->normalize($token);

if($isCnWord)

$this->\_position = $this->\_position - 3;

if ($token !== null)

return $token;

return null;

With the help of chinese.php we can search Japanese and Chinese in kb. And also we must add codes as below before creating an index and searching.

require\_once 'chinese.php';

Zend\_Search\_Lucene\_Analysis\_Analyzer::setDefault(new CN\_Lucene\_Analyzer());

**7.Is Zend Lucene need downtime?**

By using Zend Lucene , we don’t need any downtime. When add a new article we can add it to index at the same time, If we edit an article, we need to delete old document and update index with new one .

[http_www.zend.com_community_downloads]: http://www.zend.com/community/downloads