Skip to main content

Simple Script to test Sharding on MongoDB

Sharding, eh? There's so many questions on a daily basis about sharding -
  • What is sharding?
  • How do I do shard?
  • When do I do shard?
    • How do I know I need to shard?
  • How many shards do I need?
  • What shard key should I use?
  • Can I change my shard key?
  • What's a hotspot?
  • How many shards do I need?
  • Do I have a replica set within a shard?
  • etc
 and everyone is unique with a different use-case so the answer isn't always the same.

Here's the official documents page (on sharding) and Kristina's blog, which is simply excellent on so many levels - I recommend reading both links (btw, it'll take a while :). Kristina uses some awesome analogies to explain sharding.

This blog post isn't about the technicalities of sharding, there are much more intelligent people than me who can explain that. I wrote a simple script to learn a bit more sharding and for reproducing issues and I thought I'd share it. It's written in bash because I didn't want to worry about dependencies :)

After you run the script, you should be able to run  
 $ mongo twitter --eval 'sh.status()'
from your local shell and you should see something like the following indicating that you have now created a sharded cluster with a sharded database "twitter", a sharded collection "tweets" 
 MongoDB shell version: 2.0.6  
 connecting to: twitter  
 --- Sharding Status ---  
  sharding version: { "_id" : 1, "version" : 3 }  
  shards:  
   { "_id" : "shard0000", "host" : "localhost:10000" }  
   { "_id" : "shard0001", "host" : "localhost:10001" }  
   { "_id" : "shard0002", "host" : "localhost:10002" }  
  databases:  
   { "_id" : "admin", "partitioned" : false, "primary" : "config" }  
   { "_id" : "twitter", "partitioned" : true, "primary" : "shard0000" }  
     twitter.tweets chunks:  
         shard0000  1  
       { "query" : { $minKey : 1 }, "max_id" : { $minKey : 1 } } -->> { "query" : { $maxKey : 1 }, "max_id" : { $maxKey : 1 } } on : shard0000 { "t" : 1000, "i" : 0 }

Hopefully it's of interest or help to someone :)
Labels:

Comments

Post a Comment

Popular posts from this blog

LinkedIn Emails

Receiving mails via LinkedIn is an interesting experience. For example, how many folk actually personalise "contact requests" - from what I see, less than 1%. I typically try to because I think it shows some thought has gone into the request and it's friendly, but then "manners" on the Internet is a very different thing to the real world, right ;-) Anyway, to the point of the blog post. In early November (2012), whilst I was preparing my Security Onion presentation for IrissCon  (why did I bother when my MBP died on-stage), I received a very interesting and personal email via LinkedIn. The email came from a "Senior International Belief Instigator" (let's call him the SIBI - to save me typing) at Riot Games and the email was literally awesome, it hit many of the key points that you'd hope for in a recruiter email but it also had a wonderful tone. In my ignorance, I knew of League of Legends but not Riot (yes, I am embarrassed by that). I r...
9 comments
Read more

WAF versus DPI Firewall

This is a question, I've frequently been asked in recent years and in the last month, o n one of the internal mailing lists, in my old company, the following question was posted – In simple terms, what tasks is a Web Application Firewall (WAF) able to do that a Deep Inspection Firewall can't and why ? by one of my colleagues. Many of you may be surprised (I know I was initially) but this question still comes up an awful lot. Having answered the email (as a warning, I went into a lot of detail and plugged the awesome Security Onion ), I was requested to write a technical blog on the subject, but as I left the company soon after, the blog was never published. Therefore, to save me answering the question again, I thought I’d publish it so I can just reference the link in future J
6 comments
Read more

Doing The GSE

So, as many folks know, I went to Orlando towards the end of March to attempt the GSE lab. Both before and afterwards, I received several questions about the GSE :) Therefore, instead of destroying my fingers and typing multiple individual respones, I figured I'd write a short blog on my experiences with the lab section, whilst my thoughts on the written section can be found here . Apologies, this post started off short. Firstly, let me say, that once I overcome the initial nerves (I was bricking it on the first morning), I had a great time. @Chris_Mohan and @asho_relaxo both told me that I'd have fun but I didn't believe them (in fairness, they're not trustworthy characters). Most folk enjoy the first day the most, but I loved the second morning, it was a blast, especially when you come back to that problem that you couldn't figure out and then you nail it :)
6 comments
Read more