【Mongodb】对 shard 进行大量数据拆分测试

刺骨的言语ヽ痛彻心扉 2022-01-26 12:07 191阅读 0赞

前面介绍了 [如何搭建mongodb sharding 集群][mongodb sharding] ，本文对shard进行大量数据拆分测试，并谈了对于片键的选择注意事项，（可能不全，希望指教）

**1 激活test数据库的分片功能。**

mongos> db.runCommand(\{"enablesharding": "test"\})

\{ "ok" : 1 \}

**2 查看整个sharding 的架构**

mongos> db.runCommand(\{listshards:1\});

"shards" : \[

"\_id" : "shard0000",

"host" : "10.250.7.225:27018"

\},

"\_id" : "shard0001",

"host" : "10.250.7.249:27019"

\},

"\_id" : "shard0002",

"host" : "10.250.7.241:27020"

\],

"ok" : 1

mongos> printShardingStatus();

\--- Sharding Status ---

sharding version: \{ "\_id" : 1, "version" : 3 \}

shards:

\{ "\_id" : "shard0000", "host" : "10.250.7.225:27018" \}

\{ "\_id" : "shard0001", "host" : "10.250.7.249:27019" \}

\{ "\_id" : "shard0002", "host" : "10.250.7.241:27020" \}

databases:

\{ "\_id" : "admin", "partitioned" : false, "primary" : "config" \}

\{ "\_id" : "test", "partitioned" : true, "primary" : "shard0000" \}

**Note：test 数据库的 partitioned 为true 意味开启了sharding 功能**。

3 **激活test数据库中的集合sharding功能，并指定片键为"\_id"，索引唯一。**

mongos> db.runCommand(\{shardcollection:'test.yql',key:\{\_id:1\}, unique : true\});

\{ "collectionsharded" : "test.yql", "ok" : 1 \}

插入17条数据，并查看yql的状况

mongos> db.yql.stats();

"sharded" : true,

"flags" : 1,

"ns" : "test.yql",

"count" : 17,

"numExtents" : 1,

"size" : 1616,

"storageSize" : 8192,

"totalIndexSize" : 8176,

"indexSizes" : \{

"\_id\_" : 8176

\},

"avgObjSize" : 95.05882352941177,

"nindexes" : 1,

"nchunks" : 1,

"shards" : \{

**"shard0000" : \{ --表示yql被拆分到了10.250.7.225这台机器上了**

"ns" : "test.yql",

"count" : 17,

"size" : 1616,

"avgObjSize" : 95.05882352941177,

"storageSize" : 8192,

"numExtents" : 1,

"nindexes" : 1,

"lastExtentSize" : 8192,

"paddingFactor" : 1,

"flags" : 1,

"totalIndexSize" : 8176,

"indexSizes" : \{

"\_id\_" : 8176

\},

"ok" : 1

\},

"ok" : 1

mongos>

**4 使用 pymongodb 向test.yql插入200w的数据**

\[mongodb@rac4 pymongo\]$ python

Python 2.4.3 (\#1, Jan 21 2009, 01:11:33)

\[GCC 4.1.2 20071124 (Red Hat 4.1.2-42)\] on linux2

Type "help", "copyright", "credits" or "license" for more information.

>>>

>>> import pymongo

>>>conn = pymongo.Connection(host="127.0.0.1")

>>> db=conn.test

>>> for i in xrange(2000000):

... val=dict(val="yangql"+str(i))

... db.yql.insert(val)

ObjectId('4eb2aa5940643e5bb61e847b')

ObjectId('4eb2aa5940643e5bb61e847c')

ObjectId('4eb2aa5940643e5bb61e847d')

ObjectId('4eb2aa5940643e5bb61e847e')

ObjectId('4eb2aa5940643e5bb61e847f')

....省略....

**5 再次查看,collection yql在整个sharding的分布状况：这次collection yql 被拆分到了三个不同的shard上面。**

mongos> db.yql.stats();

"sharded" : true,

"flags" : 1,

"ns" : "test.yql",

"count" : 2000017,

"numExtents" : 23,

**"size" : 92001336,**

"storageSize" : 146833408,

"totalIndexSize" : 64917440,

"indexSizes" : \{

"\_id\_" : 64917440

\},

"avgObjSize" : 46.00027699764552,

"nindexes" : 1,

"nchunks" : 4,

"shards" : \{

"shard0000" : \{

"ns" : "test.yql",

"count" : 29448,

**"size" : 1296204,**

"avgObjSize" : 44.016707416462914,

"storageSize" : 2793472,

"numExtents" : 5,

"nindexes" : 1,

"lastExtentSize" : 2097152,

"paddingFactor" : 1,

"flags" : 1,

"totalIndexSize" : 964768,

"indexSizes" : \{

"\_id\_" : 964768

\},

"ok" : 1

\},

"shard0001" : \{

"ns" : "test.yql",

"count" : 1942524,

"size" : 89471128,

"avgObjSize" : 46.05921368281679,

**"storageSize" : 141246464, --这里shard0001上面的数据比较多，和片键的选择有关。**

"numExtents" : 13,

"nindexes" : 1,

"lastExtentSize" : 30072832,

"paddingFactor" : 1,

"flags" : 1,

"totalIndexSize" : 63036960,

"indexSizes" : \{

"\_id\_" : 63036960

\},

"ok" : 1

\},

"shard0002" : \{

"ns" : "test.yql",

"count" : 28045,

**"size" : 1234004,**

"avgObjSize" : 44.00085576751649,

"storageSize" : 2793472,

"numExtents" : 5,

"nindexes" : 1,

"lastExtentSize" : 2097152,

"paddingFactor" : 1,

"flags" : 1,

"totalIndexSize" : 915712,

"indexSizes" : \{

"\_id\_" : 915712

\},

"ok" : 1

\},

"ok" : 1

**从下面的查询结果可以看出，集合yql 根据片键的分布：**

shard0000 \_id:\{$minKey:1\} -->>"4eb298b3adbd9673afee95e3"

shard0000 \_id:4eb298b3adbd9673afee95e3"-->>"4eb2a64640643e5bb60072f7"

shard0002 \_id: "4eb2a64640643e5bb60072f7-->>"4eb2a65340643e5bb600e084"

shard0001 \_id:4eb2a65340643e5bb600e084"-->>\{ $maxKey:1\}

mongos> printShardingStatus();

\--- Sharding Status ---

sharding version: \{ "\_id" : 1, "version" : 3 \}

shards:

\{ "\_id" : "shard0000", "host" : "10.250.7.225:27018" \}

\{ "\_id" : "shard0001", "host" : "10.250.7.249:27019" \}

\{ "\_id" : "shard0002", "host" : "10.250.7.241:27020" \}

databases:

\{ "\_id" : "admin", "partitioned" : false, "primary" : "config" \}

\{ "\_id" : "test", "partitioned" : true, "primary" : "shard0000" \}

test.yql chunks:

shard0000 2

shard0002 1

shard0001 1

\{ "\_id" : \{ $minKey : 1 \} \} -->> \{ "\_id" : ObjectId("4eb298b3adbd9673afee95e3") \} on : shard0000 \{ "t" : 2000, "i" : 1 \}

\{ "\_id" : ObjectId("4eb298b3adbd9673afee95e3") \} -->> \{ "\_id" : ObjectId("4eb2a64640643e5bb60072f7") \} on : shard0000 \{ "t" : 1000, "i" : 3 \}

\{ "\_id" : ObjectId("4eb2a64640643e5bb60072f7") \} -->> \{ "\_id" : ObjectId("4eb2a65340643e5bb600e084") \} on : shard0002 \{ "t" : 3000, "i" : 1 \}

\{ "\_id" : ObjectId("4eb2a65340643e5bb600e084") \} -->> \{ "\_id" : \{ $maxKey : 1 \} \} on : shard0001 \{ "t" : 3000, "i" : 0 \}

mongos>

在配置的时候，我们只是指定了片键，对集合的拆分则是由mongos来进行的，即集群自动进行数据拆分，并进行负载均衡！对于片键的选择会影响到集合在shard的分布，对于本文中的例子：

选择\_id 为片键，分了四个区间：

\[\{$minKey:1\},"4eb298b3adbd9673afee95e3" \]

\["4eb298b3adbd9673afee95e3","4eb2a64640643e5bb60072f7"\]

\["4eb2a64640643e5bb60072f7","4eb2a65340643e5bb600e084"\]

\["4eb2a65340643e5bb600e084",\{$maxKey:1\}\]

随着数据的继续插入，再进行拆分数据插入的数据依然会插入到shard0001服务器上的，对于上面的例子再次插入200w的数据：

查看yql的分片分布：shard0000 和shard0002上面存储的数据没有变化，所有插入的数据全部落入到shard0001上面。

mongos> db.yql.stats();

"sharded" : true,

"flags" : 1,

"ns" : "test.yql",

"count" : 4000017,

"numExtents" : 26,

"size" : 184000936,

"storageSize" : 278208512,

"totalIndexSize" : 129802176,

"indexSizes" : \{

"\_id\_" : 129802176

\},

"avgObjSize" : 46.00003849983638,

"nindexes" : 1,

"nchunks" : 4,

"shards" : \{

"shard0000" : \{

"ns" : "test.yql",

"count" : 29448,

**"size" : 1296204**,

"avgObjSize" : 44.016707416462914,

"storageSize" : 2793472,

"numExtents" : 5,

"nindexes" : 1,

"lastExtentSize" : 2097152,

"paddingFactor" : 1,

"flags" : 1,

"totalIndexSize" : 964768,

"indexSizes" : \{

"\_id\_" : 964768

\},

"ok" : 1

\},

"shard0001" : \{

"ns" : "test.yql",

"count" : 3942524,

**"size" : 181470728,**

"avgObjSize" : 46.02907376086994,

"storageSize" : 272621568,

"numExtents" : 16,

"nindexes" : 1,

"lastExtentSize" : 51974144,

"paddingFactor" : 1,

"flags" : 1,

"totalIndexSize" : 127921696,

"indexSizes" : \{

"\_id\_" : 127921696

\},

"ok" : 1

\},

"shard0002" : \{

"ns" : "test.yql",

"count" : 28045,

**"size" : 1234004,**

"avgObjSize" : 44.00085576751649,

"storageSize" : 2793472,

"numExtents" : 5,

"nindexes" : 1,

"lastExtentSize" : 2097152,

"paddingFactor" : 1,

"flags" : 1,

"totalIndexSize" : 915712,

"indexSizes" : \{

"\_id\_" : 915712

\},

"ok" : 1

\},

"ok" : 1

mongos>

**这样不符合sharding的初衷：负载均衡，最终也会导致shard0001 上面的磁盘空间和内存不足。**

Note：如果写入负载比较高，要分散负载，此时必须考虑选择**均匀分布**的片键。不能在只有几个值的片键，比如对于status 有四个值，A,B,C,D.mongos 不会创建多于4个块。选择片键时，除了均匀分布也要有个**增长上线即最大值**，否则就会出现上面的情况，违背了sharding的初衷。

[mongodb sharding]: http://space.itpub.net/22664653/viewspace-710195