kettle spoon 在Linux中通过Kitchen和Pan以命令行方式执行kettle的Job和Transformation

迈不过友情╰ 2023-01-07 04:59 383阅读 0赞

## 简述逻辑： ##

在windows界面打开kettle  进行脚本编写 ，编写好后把kettle的安装压缩包上传到linux服务器 解压 ，在目录下通过Kitchen.sh执行作业，Pan执行转换

## 1. 准备工作 ##

一个简单的job，一个简单的trans。
    
    本处为了方便和效果易见，job和trans都生成文件。

**trans**：读取download目录下的所有文件名，输出为文件。【界面情况下测试成功】  
![这里写图片描述][SouthEast]  
成功生成目标文件：  
![这里写图片描述][SouthEast 1]  
**job**：创建文件。【界面模式测试执行成功】  
![这里写图片描述][SouthEast 2]  
执行结果：  
![这里写图片描述][SouthEast 3]  
把界面执行测试结果文件删除，以免影响观察。

## 2. linux环境以命令行方式执行job和trans ##

Pan是用于执行trans的PDI命令行工具。
        Kitchen是用于执行作业的PDI命令行工具。

**a. Pan的命令行选项和语法**  
语法：

pan.sh -option=value arg1 arg2

命令行参数：

<table> 
 <thead> 
  <tr> 
   <th>Switch</th> 
   <th>Purpose</th> 
  </tr> 
 </thead> 
 <tbody> 
  <tr> 
   <td>rep</td> 
   <td>Enterprise or database repository name, if you are using one</td> 
  </tr> 
  <tr> 
   <td>user</td> 
   <td>Repository username</td> 
  </tr> 
  <tr> 
   <td>pass</td> 
   <td>Repository password</td> 
  </tr> 
  <tr> 
   <td>trans</td> 
   <td>The name of the transformation (as it appears in the repository) to launch</td> 
  </tr> 
  <tr> 
   <td>dir</td> 
   <td>The repository directory that contains the transformation, including the leading slash</td> 
  </tr> 
  <tr> 
   <td>file</td> 
   <td>If you are calling a local KTR file, this is the filename, including the path if it is not in the local directory</td> 
  </tr> 
  <tr> 
   <td>level</td> 
   <td>The logging level (Basic, Detailed, Debug, Rowlevel, Error, Nothing)</td> 
  </tr> 
  <tr> 
   <td>logfile</td> 
   <td>A local filename to write log output to</td> 
  </tr> 
  <tr> 
   <td>listdir</td> 
   <td>Lists the directories in the specified repository</td> 
  </tr> 
  <tr> 
   <td>listtrans</td> 
   <td>Lists the transformations in the specified repository directory</td> 
  </tr> 
  <tr> 
   <td>listrep</td> 
   <td>Lists the available repositories</td> 
  </tr> 
  <tr> 
   <td>exprep</td> 
   <td>Exports all repository objects to one XML file</td> 
  </tr> 
  <tr> 
   <td>norep</td> 
   <td>Prevents Pan from logging into a repository. If you have set the KETTLE_REPOSITORY, KETTLE_USER, and KETTLE_PASSWORD environment variables, then this option will enable you to prevent Pan from logging into the specified repository, assuming you would like to execute a local KTR file instead.</td> 
  </tr> 
  <tr> 
   <td>safemode</td> 
   <td>Runs in safe mode, which enables extra checking</td> 
  </tr> 
  <tr> 
   <td>version</td> 
   <td>Shows the version, revision, and build date</td> 
  </tr> 
  <tr> 
   <td>param</td> 
   <td>Set a named parameter in a name=value format. For example: -param:FOO=bar</td> 
  </tr> 
  <tr> 
   <td>listparam</td> 
   <td>List information about the defined named parameters in the specified transformation.</td> 
  </tr> 
  <tr> 
   <td>maxloglines</td> 
   <td>The maximum number of log lines that are kept internally by PDI. Set to 0 to keep all rows (default)</td> 
  </tr> 
  <tr> 
   <td>maxlogtimeout</td> 
   <td>The maximum age (in minutes) of a log line while being kept internally by PDI. Set to 0 to keep all rows indefinitely (default)</td> 
  </tr> 
 </tbody> 
</table>

示例：

sh pan.sh -rep=initech_pdi_repo -user=pgibbons -pass=lumburghsux -trans=TPS_reports_2011

*  1

本地trans调用示例：

./pan.sh -file=/home/hadoop/workplace/kettle/trans/test_cml.ktr -norep

*  1

**b.Kitchen的命令行参数及语法：**

语法与Pan一样，参数有点不同。

<table> 
 <thead> 
  <tr> 
   <th>Switch</th> 
   <th>urpose</th> 
  </tr> 
 </thead> 
 <tbody> 
  <tr> 
   <td>rep</td> 
   <td>Enterprise or database repository name, if you are using one</td> 
  </tr> 
  <tr> 
   <td>user</td> 
   <td>Repository username</td> 
  </tr> 
  <tr> 
   <td>pass Repository</td> 
   <td>password</td> 
  </tr> 
  <tr> 
   <td>job</td> 
   <td>The name of the job (as it appears in the repository) to launch</td> 
  </tr> 
  <tr> 
   <td>dir</td> 
   <td>The repository directory that contains the job, including the leading slash</td> 
  </tr> 
  <tr> 
   <td>file</td> 
   <td>If you are calling a local KJB file, this is the filename, including the path if it is not in the local directory</td> 
  </tr> 
  <tr> 
   <td>level</td> 
   <td>The logging level (Basic, Detailed, Debug, Rowlevel, Error, Nothing)</td> 
  </tr> 
  <tr> 
   <td>logfile</td> 
   <td>A local filename to write log output to</td> 
  </tr> 
  <tr> 
   <td>listdir</td> 
   <td>Lists the sub-directories within the specified repository directory</td> 
  </tr> 
  <tr> 
   <td>listjob</td> 
   <td>Lists the jobs in the specified repository directory</td> 
  </tr> 
  <tr> 
   <td>listrep</td> 
   <td>Lists the available repositories</td> 
  </tr> 
  <tr> 
   <td>export</td> 
   <td>Exports all linked resources of the specified job. The argument is the name of a ZIP file.</td> 
  </tr> 
  <tr> 
   <td>norep</td> 
   <td>Prevents Kitchen from logging into a repository. If you have set the KETTLE_REPOSITORY, KETTLE_USER, and KETTLE_PASSWORD environment variables, then this option will enable you to prevent Kitchen from logging into the specified repository, assuming you would like to execute a local KTR file instead.</td> 
  </tr> 
  <tr> 
   <td>version</td> 
   <td>Shows the version, revision, and build date</td> 
  </tr> 
  <tr> 
   <td>param</td> 
   <td>Set a named parameter in a name=value format. For example: -param:FOO=bar</td> 
  </tr> 
  <tr> 
   <td>listparam</td> 
   <td>List information about the defined named parameters in the specified job.</td> 
  </tr> 
  <tr> 
   <td>maxloglines</td> 
   <td>The maximum number of log lines that are kept internally by PDI. Set to 0 to keep all rows (default)</td> 
  </tr> 
  <tr> 
   <td>maxlogtimeout</td> 
   <td>The maximum age (in minutes) of a log line while being kept internally by PDI. Set to 0 to keep all rows indefinitely (default)</td> 
  </tr> 
 </tbody> 
</table>

执行本地job的命令行语句：

/home/kettle/data-integration/kitchen.sh -file=/home/kettle/transition/move.kjb -log=log.log

*  1

形式：

$kitchen路径 -file=$job路径 log=$log路径

*  1

**调用pan结果：**  
![这里写图片描述][SouthEast 4]  
调用kitchen结果：  
![这里写图片描述][SouthEast 5]

## 3.个人常用命令选项 ##

由于我当前的工作环境都是执行本地的job和trans文件，所以常用的命令选项有：

<table> 
 <thead> 
  <tr> 
   <th>命令</th> 
   <th>描述</th> 
  </tr> 
 </thead> 
 <tbody> 
  <tr> 
   <td>-file</td> 
   <td>job或trans文件路径</td> 
  </tr> 
  <tr> 
   <td>-norep</td> 
   <td>标明不是资源库里的文件</td> 
  </tr> 
  <tr> 
   <td>-param</td> 
   <td>参数设置</td> 
  </tr> 
  <tr> 
   <td>-logfile</td> 
   <td>log输出文件名</td> 
  </tr> 
  <tr> 
   <td>-level</td> 
   <td>log级别 (Basic, Detailed, Debug, Rowlevel, Error, Nothing)</td> 
  </tr> 
 </tbody> 
</table>

[SouthEast]: /images/20221119/8ac3abceb8bd458d93d96484ead6bc45.png
[SouthEast 1]: https://img-blog.csdn.net/20170925111501431?watermark/2/text/aHR0cDovL2Jsb2cuY3Nkbi5uZXQvenVvbG92ZWZ1/font/5a6L5L2T/fontsize/400/fill/I0JBQkFCMA==/dissolve/70/gravity/SouthEast
[SouthEast 2]: https://img-blog.csdn.net/20170925111719794?watermark/2/text/aHR0cDovL2Jsb2cuY3Nkbi5uZXQvenVvbG92ZWZ1/font/5a6L5L2T/fontsize/400/fill/I0JBQkFCMA==/dissolve/70/gravity/SouthEast
[SouthEast 3]: https://img-blog.csdn.net/20170925111741020?watermark/2/text/aHR0cDovL2Jsb2cuY3Nkbi5uZXQvenVvbG92ZWZ1/font/5a6L5L2T/fontsize/400/fill/I0JBQkFCMA==/dissolve/70/gravity/SouthEast
[SouthEast 4]: https://img-blog.csdn.net/20170925112957900?watermark/2/text/aHR0cDovL2Jsb2cuY3Nkbi5uZXQvenVvbG92ZWZ1/font/5a6L5L2T/fontsize/400/fill/I0JBQkFCMA==/dissolve/70/gravity/SouthEast
[SouthEast 5]: https://img-blog.csdn.net/20170925113023150?watermark/2/text/aHR0cDovL2Jsb2cuY3Nkbi5uZXQvenVvbG92ZWZ1/font/5a6L5L2T/fontsize/400/fill/I0JBQkFCMA==/dissolve/70/gravity/SouthEast