Thursday, August 13, 2015

Google Cloud Launches Dataflow and Cloud Pub/Sub

Google Cloud Platform announced availability of two new data processing services: Cloud Dataflow, Cloud Pub/Sub, and BigQuery.

Google said its Cloud Dataflow is specifically designed to remove the complexity of developing separate systems for batch and streaming data sources by providing a unified programming model.  It leverages MapReduce, FlumeJava, and Millwheel.

Google Cloud Pub/Sub is designed to deliver real-time and reliable messaging for applications.  Google's alpha and beta customers have already used it to deliver over a trillion messages. Cloud Pub/Sub connects your services to each other, to other Google APIs, and third parties. The idea is to integrate applications and services reliably, as well as analyze big data streams in real-time. Pricing begins at 5¢ per million message operations for sustained usage.