Skip to main content

Simple Script to test Sharding on MongoDB

Sharding, eh? There's so many questions on a daily basis about sharding -
  • What is sharding?
  • How do I do shard?
  • When do I do shard?
    • How do I know I need to shard?
  • How many shards do I need?
  • What shard key should I use?
  • Can I change my shard key?
  • What's a hotspot?
  • How many shards do I need?
  • Do I have a replica set within a shard?
  • etc
 and everyone is unique with a different use-case so the answer isn't always the same.

Here's the official documents page (on sharding) and Kristina's blog, which is simply excellent on so many levels - I recommend reading both links (btw, it'll take a while :). Kristina uses some awesome analogies to explain sharding.

This blog post isn't about the technicalities of sharding, there are much more intelligent people than me who can explain that. I wrote a simple script to learn a bit more sharding and for reproducing issues and I thought I'd share it. It's written in bash because I didn't want to worry about dependencies :)

After you run the script, you should be able to run 
 $ mongo twitter --eval 'sh.status()'  
from your local shell and you should see something like the following indicating that you have now created a sharded cluster with a sharded database "twitter", a sharded collection "tweets"
 MongoDB shell version: 2.0.6  
 connecting to: twitter  
 --- Sharding Status ---  
  sharding version: { "_id" : 1, "version" : 3 }  
  shards:  
   { "_id" : "shard0000", "host" : "localhost:10000" }  
   { "_id" : "shard0001", "host" : "localhost:10001" }  
   { "_id" : "shard0002", "host" : "localhost:10002" }  
  databases:  
   { "_id" : "admin", "partitioned" : false, "primary" : "config" }  
   { "_id" : "twitter", "partitioned" : true, "primary" : "shard0000" }  
     twitter.tweets chunks:  
         shard0000  1  
       { "query" : { $minKey : 1 }, "max_id" : { $minKey : 1 } } -->> { "query" : { $maxKey : 1 }, "max_id" : { $maxKey : 1 } } on : shard0000 { "t" : 1000, "i" : 0 }  

Hopefully it's of interest or help to someone :)

Comments

Popular posts from this blog

Being a Support Engineer @ 10gen - Part 1

There's a mis-conception around the role of a "Support Engineer".  As a clue, it's not what Urban Dictionary   says   - A person whose job is to answer calls from customers of a small- to large-sized company...... They are teathered to a their desk all day via phone headset........ phone jockeys usually hate their jobs.......they are are paid well enough..........until they completely burn out, and hate everyone.   and doesn't always involve this - Image Source: http://half-bakedbaker.blogspot.ie/2009/11/cannoli-and-broken-computer.html As you can see  here , there's lots of open roles in  10gen  and more specifically with 10gen, in  Dublin . I thought I'd write this quick blog to explain what Support Engineers actually do and why I joined 10gen as a "Support Engineer". I could be wrong but didn't Google come up with term " Site Reliability Engineer " to do away with the stigma associated with being a

What's the point of (InfoSec) Certifications?

Quite recently, my GSE was up for renewal. I'm currently in the middle of transporting my family to another continent and I've slightly more responsibilities work-wise in 2016 versus 2012. However, given the effort and study that it took to get the cert the first time (and to a lesser degree the expense), I figured it was a no-brainer to renew. For me, I've always been a huge fan of the GSE and considered it the epitome of InfoSec certifications, much like the CCIE for (Cisco) networking. Personally, I learn better by "doing" and consider it as the evidence that someone knows their stuff so the "2-day lab" element in the GSE was a both a huge goal and challenge that I was excited about. I talked about the value of "doing" when trying to learn about yourself previously here with the infamous Security Ninja and here on my own blog so there's no point in repeating myself. When I did the GSE, I absolutely loved the hands-on lab mo

Start-Up Security

After many years in Security @ Riot Games and eventually putting the "s' out there, I recently decided to jump out of my comfort circle for a new challenge and joined a   start-up   (yes, I left a comfortable, stable job in a pandemic, lunacy lol). Now that I've been here almost 6 months, I wanted to share some findings because security at a start-up is significantly different.  When you join a start-up, there's going to be so much that you can do and it will be incredibly easy to "boil the ocean", and try to fix everything. At best, this guarantees failure for the Security team, at worst, alienation from the engineering and product teams. There are some obvious quick wins that a Security team can make without slowing down iteration and innovation speed, while also reducing risk: Auth  Partner with Engineering/IT/CTO such that there's alignment on Security owning all things "auth(n|z)".  As part of this ownership, you need to be prepared to resp