awesome-bigdata
A curated link list of big data tools, databases, papers and learning resources.
- Stars
- 14.7k
- Forks
- 2.6k
- Licence
- MIT
- Language
- —
- Last update
- 6 days ago
- Type
- Curated list
What it is
This is a long, hand-picked list of links to big data frameworks, databases, tools, papers, books and videos, grouped by topic such as stream processing, distributed filesystems, and data visualization. It is a reference you browse rather than software you install. People use it to discover which tools exist in the big data space and to find starting points for further reading.
- Scan the Frameworks and Distributed Programming sections to shortlist processing engines
- Browse the Time-Series Databases or Vector Databases sections when choosing a storage option
- Work through the Interesting Papers section for background on systems like MapReduce
- Find books and videos on streaming and distributed systems
Great for
- Developers looking for an overview of big data tools to evaluate
- Students wanting a map of the big data landscape and classic papers
- Data engineers researching alternatives for a specific category like stream processing
- Anyone who wants reading material on distributed systems
Not ideal for
- People who want a single tool to install and run, since this is only a list of links
- Readers who need up-to-date reviews, because the README gives one-line descriptions and no ratings or comparisons
- Beginners looking for step-by-step tutorials, since the list points to other resources rather than teaching
How to get it
- 1
Read it online
Clone the repository locally (optional)
git clone https://github.com/oxnr/awesome-bigdata - 2
Run it
- Open the README on GitHub
- Use the table of contents to jump to a category that matches your need
- Open a few linked projects and compare their documentation
- Check each project's own repository for recent activity before relying on it
- 3
Get the most out of it
- Use the table of contents to jump straight to a category
- Check the Books and Videos sections for longer learning material
- Follow the Interesting Papers section in chronological order for historical context
- Open the linked project pages to check their license and maintenance status
Downloads
Strengths
- Broad coverage across many categories in one place
- Each entry has a short description to help you decide what to open next
- MIT licensed and widely starred, with a clear table of contents
Trade-offs
- Entries are one-line descriptions only, so the list does not judge quality or currency of each project
- Some linked projects may be outdated or no longer maintained, and the README does not mark which ones
- Links point to many external sites, so content quality varies
How it compares
| Project | Stars | Type | Level | Updated | Best for |
|---|---|---|---|---|---|
| awesome-bigdata (this) | 14.7k | Curated list | beginner | 6 days ago | Developers looking for an overview of big data tools to evaluate |
| awesome-datascience | 30.3k | Curated list | beginner | 5 days ago | Beginners who want a guided set of links to start learning data science |
| Data-Science-For-Beginners | 37.9k | Learning resource | beginner | 16 days ago | Complete beginners who want a guided, week-by-week data science course |
| akshare | 23.2k | Library | beginner | 4 days ago | Python users doing academic or personal research on Chinese financial markets |
| dash | 24.5k | Framework | intermediate | 6 days ago | Python developers who want to share data analysis as an interactive web app |
| vanna | 23.8k | Library | intermediate | 9 months ago | Developers adding a natural-language data question box to an internal app |
Stats synced from GitHub. The assessment above was drafted with AI from the project's README on 2026-10-29 and may miss recent changes — check the README before relying on a command. MIT licence: free to use, modify and distribute, including commercially, as long as you keep the copyright notice.