Big Data
- Big Query
- Pub/Sub
- Dataflow
Big Query
- Fully managed analytics data warehouse
- Query with SQL-like syntax
- Zero administration
Pub/Sub
- Messaging for GCP and more
- many to many async messaging
- Integrates with cloud Dataflow
- 1 million messages per second
- Use cases:
- Building blocks for data ingestion
- Dataflow streaming
- Push notifications
- Connect apps across GCP
Dataflow
- Use Cases:
- ETL pipelines to move, filter, enrich, shape data
- Data analysis - batch computation for continuous computation using streaming
- Orchestration - pipelines that coordinate services, including external services
Dataproc
- Creating clusters
- Scaling clusters
- Use cases:
- Analyze data, like log data, stored in cloud storage
- Data mining / analysis
- classification algos
Datalab