在Spark中自定义Kryo序列化输入输出API

向右看齐 2022-08-04 15:52 117阅读 0赞

在[Spark][]中内置支持两种系列化格式：（1）、Java serialization；（2）、Kryo serialization。在默认情况下，[Spark][]使用的是Java的ObjectOutputStream系列化框架，它支持所有继承java.io.Serializable的类系列化，虽然Java系列化非常灵活，但是它的性能不佳。然而我们可以使用Kryo 库来系列化，它相比Java serialization系列化高效，速度很快（通常比Java快10x），但是它不支持所有的系列化对象，而且要求用户注册类。

在[Spark][]中，使用Kryo系列化比使用Java系列化更明智。在shuffling和caching大量数据的情况下，使用 Kryo系列化就变得非常重要。

虽然Kryo支持对RDD的cache和shuffle，但是在Spark中不是内置就显示提供使用Kryo将数据系列化到磁盘中的输入输出API，RDD中的saveAsObjectFile和SparkContext中的objectFile方法仅仅支持使用Java系列化。所以如果我们可以使用Kryo系列化将会变得很棒！

在这篇文章中，我将讨论如何自定义Kryo系列化输出输出相关API来将数据进行读写到磁盘中。

## 写数据 ##

通常，我们使用`rdd.saveAsObjectFile` API将已经系列化的对象写入到磁盘中。下面的代码将展示如何使用我们自定义的`saveAsObjectFile`方法将已经使用kryo系列化的对象写入到磁盘中：

这个函数中参数rdd就是我们需要写的数据；path是数据保存的路径。

`KryoSerializer`是Spark内部提供的用于提供操作Kryo的类。在上述代码中，我们创建了`KryoSerializer`对象，并从`rdd.context.getConf`中获取传进来的缓存大小。

所有的objectFile 将会在HDFS上保存，我们对RDD中的每个分片进行遍历，然后将他们转换成Byte数组。

对每个splitArray，我们首先创建了kryo实例，kryo是线程不安全的，所以我们在每个map操作中单独创建。当我们调用` kryoSerializer.newKryo()`来创建新的kryo实例，他也会调用我们自定义的KryoRegistrator。

一旦我们拥有kryo实例，我们就可以创建kryo输出对象，然后我们将类信息和对象写入到那个输出对象中。

我们在创建byteWritable的时候，包装了bytearray，然后保存成Sequence文件。使用那些代码我们就可以将Kryo对象写入到磁盘中。完整代码如下：

<table style="border-spacing:0px; font-size:1em!important; border:0px!important; width:auto!important; margin:0px!important; padding:0px!important; outline:0px!important; float:none!important; vertical-align:baseline!important; position:static!important; left:auto!important; top:auto!important; right:auto!important; bottom:auto!important; height:auto!important; line-height:1.1em!important; direction:ltr!important; border-collapse:collapse!important; background:none!important"> 
 <tbody style="margin:0px!important; padding:0px!important; border:0px!important; outline:0px!important; float:none!important; vertical-align:baseline!important; position:static!important; left:auto!important; top:auto!important; right:auto!important; bottom:auto!important; height:auto!important; width:auto!important; line-height:1.1em!important; font-size:1em!important; direction:ltr!important; background:none!important"> 
  <tr style="margin:0px!important; padding:0px!important; border:0px!important; outline:0px!important; float:none!important; vertical-align:baseline!important; position:static!important; left:auto!important; top:auto!important; right:auto!important; bottom:auto!important; height:auto!important; width:auto!important; line-height:1.1em!important; font-size:1em!important; direction:ltr!important; background:none!important"> 
   <td style="border:0px!important; padding:0px!important; margin:0px!important; outline:0px!important; float:none!important; vertical-align:top!important; position:static!important; left:auto!important; top:auto!important; right:auto!important; bottom:auto!important; height:auto!important; width:3em!important; line-height:1.1em!important; font-family:Consolas,'Bitstream Vera Sans Mono','Courier New',Courier,monospace!important; font-size:1em!important; direction:ltr!important; color:rgb(120,120,120)!important"> <code style="color:rgb(221,17,68); white-space:nowrap; padding:0px 0.3em 0px 0px!important; font-family:Consolas,'Bitstream Vera Sans Mono','Courier New',Courier,monospace!important; border:0px!important; font-size:1em!important; margin:0px!important; outline:0px!important; text-align:right!important; float:none!important; vertical-align:baseline!important; position:static!important; left:auto!important; top:auto!important; right:auto!important; bottom:auto!important; height:auto!important; width:2.7em!important; line-height:1.1em!important; direction:ltr!important; display:block!important; background:none!important">11</code></td> 
   <td style="margin:0px!important; border-width:0px 0px 0px 1px!important; padding:0px 0px 0px 0.5em!important; border-left-style:solid!important; border-left-color:rgb(212,208,200)!important; outline:0px!important; float:none!important; vertical-align:top!important; position:static!important; left:auto!important; top:auto!important; right:auto!important; bottom:auto!important; height:auto!important; width:auto!important; line-height:1.1em!important; font-family:Consolas,'Bitstream Vera Sans Mono','Courier New',Courier,monospace!important; font-size:1em!important; direction:ltr!important; background:none!important"> <code style="color:rgb(221,17,68); white-space:nowrap; padding:0px!important; font-family:Consolas,'Bitstream Vera Sans Mono','Courier New',Courier,monospace!important; border:0px!important; font-size:1em!important; margin:0px!important; outline:0px!important; float:none!important; vertical-align:baseline!important; position:static!important; left:auto!important; top:auto!important; right:auto!important; bottom:auto!important; height:auto!important; width:auto!important; line-height:1.1em!important; direction:ltr!important; display:inline!important; background:none!important">&nbsp;&nbsp;</code><code style="white-space:nowrap; padding:0px!important; color:rgb(127,0,85)!important; font-family:Consolas,'Bitstream Vera Sans Mono','Courier New',Courier,monospace!important; border:0px!important; font-size:1em!important; margin:0px!important; outline:0px!important; float:none!important; vertical-align:baseline!important; position:static!important; left:auto!important; top:auto!important; right:auto!important; bottom:auto!important; height:auto!important; width:auto!important; line-height:1.1em!important; font-weight:bold!important; direction:ltr!important; display:inline!important; background:none!important">def</code>&nbsp;<code style="white-space:nowrap; padding:0px!important; font-family:Consolas,'Bitstream Vera Sans Mono','Courier New',Courier,monospace!important; border:0px!important; font-size:1em!important; margin:0px!important; outline:0px!important; float:none!important; vertical-align:baseline!important; position:static!important; left:auto!important; top:auto!important; right:auto!important; bottom:auto!important; height:auto!important; width:auto!important; line-height:1.1em!important; direction:ltr!important; display:inline!important; background:none!important">saveAsObjectFile[T</code><code style="white-space:nowrap; padding:0px!important; color:rgb(127,0,85)!important; font-family:Consolas,'Bitstream Vera Sans Mono','Courier New',Courier,monospace!important; border:0px!important; font-size:1em!important; margin:0px!important; outline:0px!important; float:none!important; vertical-align:baseline!important; position:static!important; left:auto!important; top:auto!important; right:auto!important; bottom:auto!important; height:auto!important; width:auto!important; line-height:1.1em!important; font-weight:bold!important; direction:ltr!important; display:inline!important; background:none!important">:</code>&nbsp;<code style="white-space:nowrap; padding:0px!important; font-family:Consolas,'Bitstream Vera Sans Mono','Courier New',Courier,monospace!important; border:0px!important; font-size:1em!important; margin:0px!important; outline:0px!important; float:none!important; vertical-align:baseline!important; position:static!important; left:auto!important; top:auto!important; right:auto!important; bottom:auto!important; height:auto!important; width:auto!important; line-height:1.1em!important; direction:ltr!important; display:inline!important; background:none!important">ClassTag](rdd</code><code style="white-space:nowrap; padding:0px!important; color:rgb(127,0,85)!important; font-family:Consolas,'Bitstream Vera Sans Mono','Courier New',Courier,monospace!important; border:0px!important; font-size:1em!important; margin:0px!important; outline:0px!important; float:none!important; vertical-align:baseline!important; position:static!important; left:auto!important; top:auto!important; right:auto!important; bottom:auto!important; height:auto!important; width:auto!important; line-height:1.1em!important; font-weight:bold!important; direction:ltr!important; display:inline!important; background:none!important">:</code>&nbsp;<code style="white-space:nowrap; padding:0px!important; font-family:Consolas,'Bitstream Vera Sans Mono','Courier New',Courier,monospace!important; border:0px!important; font-size:1em!important; margin:0px!important; outline:0px!important; float:none!important; vertical-align:baseline!important; position:static!important; left:auto!important; top:auto!important; right:auto!important; bottom:auto!important; height:auto!important; width:auto!important; line-height:1.1em!important; direction:ltr!important; display:inline!important; background:none!important">RDD[T], path</code><code style="white-space:nowrap; padding:0px!important; color:rgb(127,0,85)!important; font-family:Consolas,'Bitstream Vera Sans Mono','Courier New',Courier,monospace!important; border:0px!important; font-size:1em!important; margin:0px!important; outline:0px!important; float:none!important; vertical-align:baseline!important; position:static!important; left:auto!important; top:auto!important; right:auto!important; bottom:auto!important; height:auto!important; width:auto!important; line-height:1.1em!important; font-weight:bold!important; direction:ltr!important; display:inline!important; background:none!important">:</code>&nbsp;<code style="white-space:nowrap; padding:0px!important; font-family:Consolas,'Bitstream Vera Sans Mono','Courier New',Courier,monospace!important; border:0px!important; font-size:1em!important; margin:0px!important; outline:0px!important; float:none!important; vertical-align:baseline!important; position:static!important; left:auto!important; top:auto!important; right:auto!important; bottom:auto!important; height:auto!important; width:auto!important; line-height:1.1em!important; direction:ltr!important; display:inline!important; background:none!important">String) { </code></td> 
  </tr> 
 </tbody> 
</table>

## 读 ##

光有写没有读对我们来说仍然不完美。通常我们使用sparkContext中的objectFile API从磁盘中读取数据，这里我们使用自定义的objectFile API来读取Kryo对象文件。

<table style="border-spacing:0px; font-size:1em!important; border:0px!important; width:auto!important; margin:0px!important; padding:0px!important; outline:0px!important; float:none!important; vertical-align:baseline!important; position:static!important; left:auto!important; top:auto!important; right:auto!important; bottom:auto!important; height:auto!important; line-height:1.1em!important; direction:ltr!important; border-collapse:collapse!important; background:none!important"> 
 <tbody style="margin:0px!important; padding:0px!important; border:0px!important; outline:0px!important; float:none!important; vertical-align:baseline!important; position:static!important; left:auto!important; top:auto!important; right:auto!important; bottom:auto!important; height:auto!important; width:auto!important; line-height:1.1em!important; font-size:1em!important; direction:ltr!important; background:none!important"> 
  <tr style="margin:0px!important; padding:0px!important; border:0px!important; outline:0px!important; float:none!important; vertical-align:baseline!important; position:static!important; left:auto!important; top:auto!important; right:auto!important; bottom:auto!important; height:auto!important; width:auto!important; line-height:1.1em!important; font-size:1em!important; direction:ltr!important; background:none!important"> 
   <td style="border:0px!important; padding:0px!important; margin:0px!important; outline:0px!important; float:none!important; vertical-align:top!important; position:static!important; left:auto!important; top:auto!important; right:auto!important; bottom:auto!important; height:auto!important; width:3em!important; line-height:1.1em!important; font-family:Consolas,'Bitstream Vera Sans Mono','Courier New',Courier,monospace!important; font-size:1em!important; direction:ltr!important; color:rgb(120,120,120)!important"> <code style="color:rgb(221,17,68); white-space:nowrap; padding:0px 0.3em 0px 0px!important; font-family:Consolas,'Bitstream Vera Sans Mono','Courier New',Courier,monospace!important; border:0px!important; font-size:1em!important; margin:0px!important; outline:0px!important; text-align:right!important; float:none!important; vertical-align:baseline!important; position:static!important; left:auto!important; top:auto!important; right:auto!important; bottom:auto!important; height:auto!important; width:2.7em!important; line-height:1.1em!important; direction:ltr!important; display:block!important; background:none!important">01</code></td> 
   <td style="margin:0px!important; border-width:0px 0px 0px 1px!important; padding:0px 0px 0px 0.5em!important; border-left-style:solid!important; border-left-color:rgb(212,208,200)!important; outline:0px!important; float:none!important; vertical-align:top!important; position:static!important; left:auto!important; top:auto!important; right:auto!important; bottom:auto!important; height:auto!important; width:auto!important; line-height:1.1em!important; font-family:Consolas,'Bitstream Vera Sans Mono','Courier New',Courier,monospace!important; font-size:1em!important; direction:ltr!important; background:none!important"> <code style="white-space:nowrap; padding:0px!important; color:rgb(127,0,85)!important; font-family:Consolas,'Bitstream Vera Sans Mono','Courier New',Courier,monospace!important; border:0px!important; font-size:1em!important; margin:0px!important; outline:0px!important; float:none!important; vertical-align:baseline!important; position:static!important; left:auto!important; top:auto!important; right:auto!important; bottom:auto!important; height:auto!important; width:auto!important; line-height:1.1em!important; font-weight:bold!important; direction:ltr!important; display:inline!important; background:none!important">def</code>&nbsp;<code style="white-space:nowrap; padding:0px!important; font-family:Consolas,'Bitstream Vera Sans Mono','Courier New',Courier,monospace!important; border:0px!important; font-size:1em!important; margin:0px!important; outline:0px!important; float:none!important; vertical-align:baseline!important; position:static!important; left:auto!important; top:auto!important; right:auto!important; bottom:auto!important; height:auto!important; width:auto!important; line-height:1.1em!important; direction:ltr!important; display:inline!important; background:none!important">objectFile[T](sc</code><code style="white-space:nowrap; padding:0px!important; color:rgb(127,0,85)!important; font-family:Consolas,'Bitstream Vera Sans Mono','Courier New',Courier,monospace!important; border:0px!important; font-size:1em!important; margin:0px!important; outline:0px!important; float:none!important; vertical-align:baseline!important; position:static!important; left:auto!important; top:auto!important; right:auto!important; bottom:auto!important; height:auto!important; width:auto!important; line-height:1.1em!important; font-weight:bold!important; direction:ltr!important; display:inline!important; background:none!important">:</code>&nbsp;<code style="white-space:nowrap; padding:0px!important; font-family:Consolas,'Bitstream Vera Sans Mono','Courier New',Courier,monospace!important; border:0px!important; font-size:1em!important; margin:0px!important; outline:0px!important; float:none!important; vertical-align:baseline!important; position:static!important; left:auto!important; top:auto!important; right:auto!important; bottom:auto!important; height:auto!important; width:auto!important; line-height:1.1em!important; direction:ltr!important; display:inline!important; background:none!important">SparkContext, path</code><code style="white-space:nowrap; padding:0px!important; color:rgb(127,0,85)!important; font-family:Consolas,'Bitstream Vera Sans Mono','Courier New',Courier,monospace!important; border:0px!important; font-size:1em!important; margin:0px!important; outline:0px!important; float:none!important; vertical-align:baseline!important; position:static!important; left:auto!important; top:auto!important; right:auto!important; bottom:auto!important; height:auto!important; width:auto!important; line-height:1.1em!important; font-weight:bold!important; direction:ltr!important; display:inline!important; background:none!important">:</code>&nbsp;<code style="white-space:nowrap; padding:0px!important; font-family:Consolas,'Bitstream Vera Sans Mono','Courier New',Courier,monospace!important; border:0px!important; font-size:1em!important; margin:0px!important; outline:0px!important; float:none!important; vertical-align:baseline!important; position:static!important; left:auto!important; top:auto!important; right:auto!important; bottom:auto!important; height:auto!important; width:auto!important; line-height:1.1em!important; direction:ltr!important; display:inline!important; background:none!important">String, minPartitions</code><code style="white-space:nowrap; padding:0px!important; color:rgb(127,0,85)!important; font-family:Consolas,'Bitstream Vera Sans Mono','Courier New',Courier,monospace!important; border:0px!important; font-size:1em!important; margin:0px!important; outline:0px!important; float:none!important; vertical-align:baseline!important; position:static!important; left:auto!important; top:auto!important; right:auto!important; bottom:auto!important; height:auto!important; width:auto!important; line-height:1.1em!important; font-weight:bold!important; direction:ltr!important; display:inline!important; background:none!important">:</code>&nbsp;<code style="white-space:nowrap; padding:0px!important; font-family:Consolas,'Bitstream Vera Sans Mono','Courier New',Courier,monospace!important; border:0px!important; font-size:1em!important; margin:0px!important; outline:0px!important; float:none!important; vertical-align:baseline!important; position:static!important; left:auto!important; top:auto!important; right:auto!important; bottom:auto!important; height:auto!important; width:auto!important; line-height:1.1em!important; direction:ltr!important; display:inline!important; background:none!important">Int&nbsp;</code><code style="white-space:nowrap; padding:0px!important; color:rgb(127,0,85)!important; font-family:Consolas,'Bitstream Vera Sans Mono','Courier New',Courier,monospace!important; border:0px!important; font-size:1em!important; margin:0px!important; outline:0px!important; float:none!important; vertical-align:baseline!important; position:static!important; left:auto!important; top:auto!important; right:auto!important; bottom:auto!important; height:auto!important; width:auto!important; line-height:1.1em!important; font-weight:bold!important; direction:ltr!important; display:inline!important; background:none!important">=</code>&nbsp;<code style="white-space:nowrap; padding:0px!important; color:rgb(0,153,0)!important; font-family:Consolas,'Bitstream Vera Sans Mono','Courier New',Courier,monospace!important; border:0px!important; font-size:1em!important; margin:0px!important; outline:0px!important; float:none!important; vertical-align:baseline!important; position:static!important; left:auto!important; top:auto!important; right:auto!important; bottom:auto!important; height:auto!important; width:auto!important; line-height:1.1em!important; direction:ltr!important; display:inline!important; background:none!important">1</code><code style="white-space:nowrap; padding:0px!important; font-family:Consolas,'Bitstream Vera Sans Mono','Courier New',Courier,monospace!important; border:0px!important; font-size:1em!important; margin:0px!important; outline:0px!important; float:none!important; vertical-align:baseline!important; position:static!important; left:auto!important; top:auto!important; right:auto!important; bottom:auto!important; height:auto!important; width:auto!important; line-height:1.1em!important; direction:ltr!important; display:inline!important; background:none!important">)</code></td> 
  </tr> 
 </tbody> 
</table>

上面的步骤和写的步骤很类似，只不过这里我们使用的是input，而不是output。我们从BytesWritable中读取bytes数据，然后使用readClassAndObject API反序列化数据。

## 如何使用 ##

下面例子使用上面介绍的两个方法来系列化和反序列化Person对象：

<table style="border-spacing:0px; font-size:1em!important; border:0px!important; width:auto!important; margin:0px!important; padding:0px!important; outline:0px!important; float:none!important; vertical-align:baseline!important; position:static!important; left:auto!important; top:auto!important; right:auto!important; bottom:auto!important; height:auto!important; line-height:1.1em!important; direction:ltr!important; border-collapse:collapse!important; background:none!important"> 
 <tbody style="margin:0px!important; padding:0px!important; border:0px!important; outline:0px!important; float:none!important; vertical-align:baseline!important; position:static!important; left:auto!important; top:auto!important; right:auto!important; bottom:auto!important; height:auto!important; width:auto!important; line-height:1.1em!important; font-size:1em!important; direction:ltr!important; background:none!important"> 
  <tr style="margin:0px!important; padding:0px!important; border:0px!important; outline:0px!important; float:none!important; vertical-align:baseline!important; position:static!important; left:auto!important; top:auto!important; right:auto!important; bottom:auto!important; height:auto!important; width:auto!important; line-height:1.1em!important; font-size:1em!important; direction:ltr!important; background:none!important"> 
   <td style="border:0px!important; padding:0px!important; margin:0px!important; outline:0px!important; float:none!important; vertical-align:top!important; position:static!important; left:auto!important; top:auto!important; right:auto!important; bottom:auto!important; height:auto!important; width:3em!important; line-height:1.1em!important; font-family:Consolas,'Bitstream Vera Sans Mono','Courier New',Courier,monospace!important; font-size:1em!important; direction:ltr!important; color:rgb(120,120,120)!important"> <code style="color:rgb(221,17,68); white-space:nowrap; padding:0px 0.3em 0px 0px!important; font-family:Consolas,'Bitstream Vera Sans Mono','Courier New',Courier,monospace!important; border:0px!important; font-size:1em!important; margin:0px!important; outline:0px!important; text-align:right!important; float:none!important; vertical-align:baseline!important; position:static!important; left:auto!important; top:auto!important; right:auto!important; bottom:auto!important; height:auto!important; width:2.7em!important; line-height:1.1em!important; direction:ltr!important; display:block!important; background:none!important">22</code></td> 
   <td style="margin:0px!important; border-width:0px 0px 0px 1px!important; padding:0px 0px 0px 0.5em!important; border-left-style:solid!important; border-left-color:rgb(212,208,200)!important; outline:0px!important; float:none!important; vertical-align:top!important; position:static!important; left:auto!important; top:auto!important; right:auto!important; bottom:auto!important; height:auto!important; width:auto!important; line-height:1.1em!important; font-family:Consolas,'Bitstream Vera Sans Mono','Courier New',Courier,monospace!important; font-size:1em!important; direction:ltr!important; background:none!important"> <code style="color:rgb(221,17,68); white-space:nowrap; padding:0px!important; font-family:Consolas,'Bitstream Vera Sans Mono','Courier New',Courier,monospace!important; border:0px!important; font-size:1em!important; margin:0px!important; outline:0px!important; float:none!important; vertical-align:baseline!important; position:static!important; left:auto!important; top:auto!important; right:auto!important; bottom:auto!important; height:auto!important; width:auto!important; line-height:1.1em!important; direction:ltr!important; display:inline!important; background:none!important">&nbsp;&nbsp;&nbsp;&nbsp;</code><code style="white-space:nowrap; padding:0px!important; color:rgb(127,0,85)!important; font-family:Consolas,'Bitstream Vera Sans Mono','Courier New',Courier,monospace!important; border:0px!important; font-size:1em!important; margin:0px!important; outline:0px!important; float:none!important; vertical-align:baseline!important; position:static!important; left:auto!important; top:auto!important; right:auto!important; bottom:auto!important; height:auto!important; width:auto!important; line-height:1.1em!important; font-weight:bold!important; direction:ltr!important; display:inline!important; background:none!important">val</code>&nbsp;<code style="white-space:nowrap; padding:0px!important; font-family:Consolas,'Bitstream Vera Sans Mono','Courier New',Courier,monospace!important; border:0px!important; font-size:1em!important; margin:0px!important; outline:0px!important; float:none!important; vertical-align:baseline!important; position:static!important; left:auto!important; top:auto!important; right:auto!important; bottom:auto!important; height:auto!important; width:auto!important; line-height:1.1em!important; direction:ltr!important; display:inline!important; background:none!important">conf&nbsp;</code><code style="white-space:nowrap; padding:0px!important; color:rgb(127,0,85)!important; font-family:Consolas,'Bitstream Vera Sans Mono','Courier New',Courier,monospace!important; border:0px!important; font-size:1em!important; margin:0px!important; outline:0px!important; float:none!important; vertical-align:baseline!important; position:static!important; left:auto!important; top:auto!important; right:auto!important; bottom:auto!important; height:auto!important; width:auto!important; line-height:1.1em!important; font-weight:bold!important; direction:ltr!important; display:inline!important; background:none!important">=</code>&nbsp;<code style="white-space:nowrap; padding:0px!important; color:rgb(127,0,85)!important; font-family:Consolas,'Bitstream Vera Sans Mono','Courier New',Courier,monospace!important; border:0px!important; font-size:1em!important; margin:0px!important; outline:0px!important; float:none!important; vertical-align:baseline!important; position:static!important; left:auto!important; top:auto!important; right:auto!important; bottom:auto!important; height:auto!important; width:auto!important; line-height:1.1em!important; font-weight:bold!important; direction:ltr!important; display:inline!important; background:none!important">new</code>&nbsp;<code style="white-space:nowrap; padding:0px!important; font-family:Consolas,'Bitstream Vera Sans Mono','Courier New',Courier,monospace!important; border:0px!important; font-size:1em!important; margin:0px!important; outline:0px!important; float:none!important; vertical-align:baseline!important; position:static!important; left:auto!important; top:auto!important; right:auto!important; bottom:auto!important; height:auto!important; width:auto!important; line-height:1.1em!important; direction:ltr!important; display:inline!important; background:none!important">SparkConf().setMaster(</code><code style="white-space:nowrap; padding:0px!important; color:rgb(42,0,255)!important; font-family:Consolas,'Bitstream Vera Sans Mono','Courier New',Courier,monospace!important; border:0px!important; font-size:1em!important; margin:0px!important; outline:0px!important; float:none!important; vertical-align:baseline!important; position:static!important; left:auto!important; top:auto!important; right:auto!important; bottom:auto!important; height:auto!important; width:auto!important; line-height:1.1em!important; direction:ltr!important; display:inline!important; background:none!important">"local"</code><code style="white-space:nowrap; padding:0px!important; font-family:Consolas,'Bitstream Vera Sans Mono','Courier New',Courier,monospace!important; border:0px!important; font-size:1em!important; margin:0px!important; outline:0px!important; float:none!important; vertical-align:baseline!important; position:static!important; left:auto!important; top:auto!important; right:auto!important; bottom:auto!important; height:auto!important; width:auto!important; line-height:1.1em!important; direction:ltr!important; display:inline!important; background:none!important">).setAppName(</code><code style="white-space:nowrap; padding:0px!important; color:rgb(42,0,255)!important; font-family:Consolas,'Bitstream Vera Sans Mono','Courier New',Courier,monospace!important; border:0px!important; font-size:1em!important; margin:0px!important; outline:0px!important; float:none!important; vertical-align:baseline!important; position:static!important; left:auto!important; top:auto!important; right:auto!important; bottom:auto!important; height:auto!important; width:auto!important; line-height:1.1em!important; direction:ltr!important; display:inline!important; background:none!important">"kryoexample"</code><code style="white-space:nowrap; padding:0px!important; font-family:Consolas,'Bitstream Vera Sans Mono','Courier New',Courier,monospace!important; border:0px!important; font-size:1em!important; margin:0px!important; outline:0px!important; float:none!important; vertical-align:baseline!important; position:static!important; left:auto!important; top:auto!important; right:auto!important; bottom:auto!important; height:auto!important; width:auto!important; line-height:1.1em!important; direction:ltr!important; display:inline!important; background:none!important">)</code></td> 
  </tr> 
 </tbody> 
</table>

<table style="border-spacing:0px; font-size:1em!important; border:0px!important; width:auto!important; margin:0px!important; padding:0px!important; outline:0px!important; float:none!important; vertical-align:baseline!important; position:static!important; left:auto!important; top:auto!important; right:auto!important; bottom:auto!important; height:auto!important; line-height:1.1em!important; direction:ltr!important; border-collapse:collapse!important; background:none!important"> 
 <tbody style="margin:0px!important; padding:0px!important; border:0px!important; outline:0px!important; float:none!important; vertical-align:baseline!important; position:static!important; left:auto!important; top:auto!important; right:auto!important; bottom:auto!important; height:auto!important; width:auto!important; line-height:1.1em!important; font-size:1em!important; direction:ltr!important; background:none!important"> 
  <tr style="margin:0px!important; padding:0px!important; border:0px!important; outline:0px!important; float:none!important; vertical-align:baseline!important; position:static!important; left:auto!important; top:auto!important; right:auto!important; bottom:auto!important; height:auto!important; width:auto!important; line-height:1.1em!important; font-size:1em!important; direction:ltr!important; background:none!important"> 
   <td style="border:0px!important; padding:0px!important; margin:0px!important; outline:0px!important; float:none!important; vertical-align:top!important; position:static!important; left:auto!important; top:auto!important; right:auto!important; bottom:auto!important; height:auto!important; width:3em!important; line-height:1.1em!important; font-family:Consolas,'Bitstream Vera Sans Mono','Courier New',Courier,monospace!important; font-size:1em!important; direction:ltr!important; color:rgb(120,120,120)!important"> <code style="color:rgb(221,17,68); white-space:nowrap; padding:0px 0.3em 0px 0px!important; font-family:Consolas,'Bitstream Vera Sans Mono','Courier New',Courier,monospace!important; border:0px!important; font-size:1em!important; margin:0px!important; outline:0px!important; text-align:right!important; float:none!important; vertical-align:baseline!important; position:static!important; left:auto!important; top:auto!important; right:auto!important; bottom:auto!important; height:auto!important; width:2.7em!important; line-height:1.1em!important; direction:ltr!important; display:block!important; background:none!important">27</code></td> 
   <td style="margin:0px!important; border-width:0px 0px 0px 1px!important; padding:0px 0px 0px 0.5em!important; border-left-style:solid!important; border-left-color:rgb(212,208,200)!important; outline:0px!important; float:none!important; vertical-align:top!important; position:static!important; left:auto!important; top:auto!important; right:auto!important; bottom:auto!important; height:auto!important; width:auto!important; line-height:1.1em!important; font-family:Consolas,'Bitstream Vera Sans Mono','Courier New',Courier,monospace!important; font-size:1em!important; direction:ltr!important; background:none!important"> <code style="color:rgb(221,17,68); white-space:nowrap; padding:0px!important; font-family:Consolas,'Bitstream Vera Sans Mono','Courier New',Courier,monospace!important; border:0px!important; font-size:1em!important; margin:0px!important; outline:0px!important; float:none!important; vertical-align:baseline!important; position:static!important; left:auto!important; top:auto!important; right:auto!important; bottom:auto!important; height:auto!important; width:auto!important; line-height:1.1em!important; direction:ltr!important; display:inline!important; background:none!important">&nbsp;&nbsp;&nbsp;&nbsp;</code><code style="white-space:nowrap; padding:0px!important; color:rgb(127,0,85)!important; font-family:Consolas,'Bitstream Vera Sans Mono','Courier New',Courier,monospace!important; border:0px!important; font-size:1em!important; margin:0px!important; outline:0px!important; float:none!important; vertical-align:baseline!important; position:static!important; left:auto!important; top:auto!important; right:auto!important; bottom:auto!important; height:auto!important; width:auto!important; line-height:1.1em!important; font-weight:bold!important; direction:ltr!important; display:inline!important; background:none!important">val</code>&nbsp;<code style="white-space:nowrap; padding:0px!important; font-family:Consolas,'Bitstream Vera Sans Mono','Courier New',Courier,monospace!important; border:0px!important; font-size:1em!important; margin:0px!important; outline:0px!important; float:none!important; vertical-align:baseline!important; position:static!important; left:auto!important; top:auto!important; right:auto!important; bottom:auto!important; height:auto!important; width:auto!important; line-height:1.1em!important; direction:ltr!important; display:inline!important; background:none!important">personList&nbsp;</code><code style="white-space:nowrap; padding:0px!important; color:rgb(127,0,85)!important; font-family:Consolas,'Bitstream Vera Sans Mono','Courier New',Courier,monospace!important; border:0px!important; font-size:1em!important; margin:0px!important; outline:0px!important; float:none!important; vertical-align:baseline!important; position:static!important; left:auto!important; top:auto!important; right:auto!important; bottom:auto!important; height:auto!important; width:auto!important; line-height:1.1em!important; font-weight:bold!important; direction:ltr!important; display:inline!important; background:none!important">=</code>&nbsp;<code style="white-space:nowrap; padding:0px!important; color:rgb(0,153,0)!important; font-family:Consolas,'Bitstream Vera Sans Mono','Courier New',Courier,monospace!important; border:0px!important; font-size:1em!important; margin:0px!important; outline:0px!important; float:none!important; vertical-align:baseline!important; position:static!important; left:auto!important; top:auto!important; right:auto!important; bottom:auto!important; height:auto!important; width:auto!important; line-height:1.1em!important; direction:ltr!important; display:inline!important; background:none!important">1</code>&nbsp;<code style="white-space:nowrap; padding:0px!important; font-family:Consolas,'Bitstream Vera Sans Mono','Courier New',Courier,monospace!important; border:0px!important; font-size:1em!important; margin:0px!important; outline:0px!important; float:none!important; vertical-align:baseline!important; position:static!important; left:auto!important; top:auto!important; right:auto!important; bottom:auto!important; height:auto!important; width:auto!important; line-height:1.1em!important; direction:ltr!important; display:inline!important; background:none!important">to&nbsp;</code><code style="white-space:nowrap; padding:0px!important; color:rgb(0,153,0)!important; font-family:Consolas,'Bitstream Vera Sans Mono','Courier New',Courier,monospace!important; border:0px!important; font-size:1em!important; margin:0px!important; outline:0px!important; float:none!important; vertical-align:baseline!important; position:static!important; left:auto!important; top:auto!important; right:auto!important; bottom:auto!important; height:auto!important; width:auto!important; line-height:1.1em!important; direction:ltr!important; display:inline!important; background:none!important">10000</code>&nbsp;<code style="white-space:nowrap; padding:0px!important; font-family:Consolas,'Bitstream Vera Sans Mono','Courier New',Courier,monospace!important; border:0px!important; font-size:1em!important; margin:0px!important; outline:0px!important; float:none!important; vertical-align:baseline!important; position:static!important; left:auto!important; top:auto!important; right:auto!important; bottom:auto!important; height:auto!important; width:auto!important; line-height:1.1em!important; direction:ltr!important; display:inline!important; background:none!important">map (value&nbsp;</code><code style="white-space:nowrap; padding:0px!important; color:rgb(127,0,85)!important; font-family:Consolas,'Bitstream Vera Sans Mono','Courier New',Courier,monospace!important; border:0px!important; font-size:1em!important; margin:0px!important; outline:0px!important; float:none!important; vertical-align:baseline!important; position:static!important; left:auto!important; top:auto!important; right:auto!important; bottom:auto!important; height:auto!important; width:auto!important; line-height:1.1em!important; font-weight:bold!important; direction:ltr!important; display:inline!important; background:none!important">=</code><code style="white-space:nowrap; padding:0px!important; font-family:Consolas,'Bitstream Vera Sans Mono','Courier New',Courier,monospace!important; border:0px!important; font-size:1em!important; margin:0px!important; outline:0px!important; float:none!important; vertical-align:baseline!important; position:static!important; left:auto!important; top:auto!important; right:auto!important; bottom:auto!important; height:auto!important; width:auto!important; line-height:1.1em!important; direction:ltr!important; display:inline!important; background:none!important">&gt;&nbsp;</code><code style="white-space:nowrap; padding:0px!important; color:rgb(127,0,85)!important; font-family:Consolas,'Bitstream Vera Sans Mono','Courier New',Courier,monospace!important; border:0px!important; font-size:1em!important; margin:0px!important; outline:0px!important; float:none!important; vertical-align:baseline!important; position:static!important; left:auto!important; top:auto!important; right:auto!important; bottom:auto!important; height:auto!important; width:auto!important; line-height:1.1em!important; font-weight:bold!important; direction:ltr!important; display:inline!important; background:none!important">new</code>&nbsp;<code style="white-space:nowrap; padding:0px!important; font-family:Consolas,'Bitstream Vera Sans Mono','Courier New',Courier,monospace!important; border:0px!important; font-size:1em!important; margin:0px!important; outline:0px!important; float:none!important; vertical-align:baseline!important; position:static!important; left:auto!important; top:auto!important; right:auto!important; bottom:auto!important; height:auto!important; width:auto!important; line-height:1.1em!important; direction:ltr!important; display:inline!important; background:none!important">Person(value +&nbsp;</code><code style="white-space:nowrap; padding:0px!important; color:rgb(42,0,255)!important; font-family:Consolas,'Bitstream Vera Sans Mono','Courier New',Courier,monospace!important; border:0px!important; font-size:1em!important; margin:0px!important; outline:0px!important; float:none!important; vertical-align:baseline!important; position:static!important; left:auto!important; top:auto!important; right:auto!important; bottom:auto!important; height:auto!important; width:auto!important; line-height:1.1em!important; direction:ltr!important; display:inline!important; background:none!important">""</code><code style="white-space:nowrap; padding:0px!important; font-family:Consolas,'Bitstream Vera Sans Mono','Courier New',Courier,monospace!important; border:0px!important; font-size:1em!important; margin:0px!important; outline:0px!important; float:none!important; vertical-align:baseline!important; position:static!important; left:auto!important; top:auto!important; right:auto!important; bottom:auto!important; height:auto!important; width:auto!important; line-height:1.1em!important; direction:ltr!important; display:inline!important; background:none!important">))</code></td> 
  </tr> 
 </tbody> 
</table>

**本博客文章除特别声明，全部都是原创！  
尊重原创，转载请注明： 转载自[过往记忆（http://www.iteblog.com/）][http_www.iteblog.com]  
本文链接地址: [《在Spark中自定义Kryo序列化输入输出API》（http://www.iteblog.com/archives/1328）][Spark_Kryo_API_http_www.iteblog.com_archives_1328]**

[Spark]: http://www.iteblog.com/archives/tag/spark
[http_www.iteblog.com]: http://www.iteblog.com/
[Spark_Kryo_API_http_www.iteblog.com_archives_1328]: http://www.iteblog.com/archives/1328