Data Stack

It is the pipeline map in the big data world. All about big data technology stack.

100 Big Data Papers

  1. 1 Architecture - Lambda 9
  2. 2 Architecture - Kappa 1
  3. 3 Architecture - SummingBird 1
  4. 4 Architecture - Data Center as a computer 1
  5. 5 Architecture - NOSQL Data Stores 1
  6. 6 Architecture - NOSQL Thesis 1
  7. 7 Architecture - Large Scale Data Management 1
  8. 8 Architecture - Eventual Consistency 3

Ingestion

  1. 1 Apache Flume 1
  2. 2 Apache Kafka 2
  3. 3 Apache Sqoop 1
  4. 4 AWS Kinesis 0
  5. 5 Fluentd 2
  6. 6 Apache Nifi 3
  7. 7 Linkedin Gobblin 1
  8. 8 rsyslog 2

Flie Format

  1. 1 Apache Avro 1
  2. 2 Apache Parquet 0
  3. 3 Apache Thrift 0
  4. 4 Protocol Buffers   0
  5. 5 SequenceFiles 0

Cluster Management Tools

  1. 1 Apache Hadoop YARN 1
  2. 2 Apache Mesos 1
  3. 3 Apache ZooKeeper 0
  4. 4 Apache Oozie 1
  5. 5 Linkedin Azkaban 1
  6. 6 Aibaba zeus 0
  7. 7 Apache Beam (incubating) 0
  8. 8 Apache Brooklyn 0

File System

  1. 1 Apache Hadoop HDFS 2
  2. 2 Amazon S3 0
  3. 3 Tachyon 2

Data Processing

  1. 1 Apache Spark SQL 2
  2. 2 Spark Streaming 0
  3. 3 Apache Hadoop MR2 0
  4. 4 Apache Hive 2
  5. 5 Apache Tez 1
  6. 6 Apache Drill 0
  7. 7 Facebook Presto 3
  8. 8 Cloudera Impala 0

Workflow Monitoring Tools

  1. 1 308 Permanent Redirect 5
  2. 2 Ambari 1
  3. 3 Ganglia 1
  4. 4 Apache NiFi 1
  5. 5 Apache Zeppelin 6

Database

  1. 1 PostgreSQL 0
  2. 2 MySQL 2
  3. 3 Oracle 2
  4. 4 Amazon RDS 2
  5. 5 MongoDB 0
  6. 6 ElasticSearch 2
  7. 7 Couchbase 0
  8. 8 Redis 1

Batch ML

  1. 1 Apache Spark R 1
  2. 2 Apache Spark MLlib 1
  3. 3 Apache Mahout 0
  4. 4 H2O 0

Data Analytics

  1. 1 Apache Lens 0

Batch Graph

  1. 1 Apache Spark Graphx 0
  2. 2 GraphLab 0
  3. 3 Giraph 0

Data Visualization

  1. 1 D3.js 4
  2. 2 Highcharts 2
  3. 3 http://echarts.baidu.com/ 5
  4. 4 Flask 3
  5. 5 Django 2
  6. 6 Grafana 1

Data Security

  1. 1 Apache Eagle 3
  2. 2 Apache Ranger 0
  3. 3 Apache Sentry 0

OLAP

  1. 1 Apache Kylin 0
Confirm
Follow page
Suggest
Message Howl Yu
Report link