Home > Terms > English (EN) > sharding

sharding

Any database that's spread across multiple machines needs some scheme to decide which machines should store a given piece of data. A sharding system makes such a decision by using the key for each row in a database table. For example, with a ten machine cluster, one way to shard is to use the last decimal digit of the index number to decide which machine to store a piece of data. Since both the storing and retrieval code follows the same rule, the system knows exactly which machine holds a specific piece of data.

The biggest challenges with sharding are splitting the data evenly across machines and dealing with changes in the cluster size. To ease the pain of these operations, more complex schemes are used to split up the data. Some of these rely on a central directory that holds the locations of particular keys. This level of indirection allows data to be moved between machines when a particular shard grows too large (to rebalance the distribution), at the cost of requiring an extra lookup in the directory for each operation. The directory's information is usually fairly small and reasonably static, though, so it's a good candidate for local caching, as long as the infrequent changes are spotted.

0
  • Part of Speech: noun
  • Synonym(s):
  • Blossary:
  • Industry/Domain: Internet
  • Category: Big data
  • Company:
  • Product:
  • Acronym-Abbreviation:
Collect to Blossary

Member comments

You have to log in to post to discussions.

Terms in the News

Billy Morgan

Sports; Snowboarding

The British snowboarder Billy Morgan has landed the sport’s first ever 1800 quadruple cork. The rider, who represented Great Britain in the 2014 Winter Olympics in Sochi, was in Livigno, Italy, when he achieved the man-oeuvre. It involves flipping four times, while body also spins with five complete rotations on a sideways or downward-facing axis. The trick ...

Marzieh Afkham

Broadcasting & receiving; News

Marzieh Afkham, who is the country’s first foreign ministry spokeswoman, will head a mission in east Asia, the state news agency reported. It is not clear to which country she will be posted as her appointment has yet to be announced officially. Afkham will only be the second female ambassador Iran has had. Under the last shah’s rule, Mehrangiz Dolatshahi, a ...

Weekly Packet

Language; Online services; Slang; Internet

Weekly Packet or "Paquete Semanal" as it is known in Cuba is a term used by Cubans to describe the information that is gathered from the internet outside of Cuba and saved onto hard drives to be transported into Cuba itself. Weekly Packets are then sold to Cuban's without internet access, allowing them to obtain information just days - and sometimes hours - after it ...

Asian Infrastructure Investment Bank (AIIB)

Banking; Investment banking

The Asian Infrastructure Investment Bank (AIIB) is an international financial institution established to address the need in Asia for infrastructure development. According to the Asian Development Bank, Asia needs $800 billion each year for roads, ports, power plants or other infrastructure projects before 2020. Originally proposed by China in 2013, a signing ...

Spartan

Online services; Internet

Spartan is the codename given to the new Microsoft Windows 10 browser that will replace Microsoft Windows Internet Explorer. The new browser will be built from the ground up and disregard any code from the IE platform. It has a new rendering engine that is built to be compatible with how the web is written today. The name Spartan is named after the ...

Featured Terms

Naveed
  • 0

    Terms

  • 5

    Blossaries

  • 1

    Followers

Industry/Domain: Materials science Category: Metallurgy

Austenitic stainless steels

Austenitic stainless steels offer excellent weldability, but austenite isn't stable at room temperature. Consequently, specific alloys must be ...

Contributor

Featured blossaries

Collaborative Lexicography

Category: Languages   1 1 Terms

Forex Jargon

Category: Business   2 19 Terms