Are you having difficulty keeping up to date on all the frequent changes and updates in the streaming data space? Then the 'Streaming Data Monthly Digest' ( updated daily!) has the solution you’re looking for. Please find below a list of web resources related to streaming data in general for January 2017.
I am daily updating this list without a focus on a particular tool be it open source or commercial. Many web resources listed below are future events such as meetups and conference talks. Related slides and videos will be added as they are made available.
Not a single streaming data processor can claim to be a silver bullet! All streaming data processors have their own strengths and weaknesses and are sweet spots for particular use cases.
January 2nd, 2017
- [Blog] Apache Flink: A New Wave to Real-time Stream Processing
- [Article] Big Data: Spark 2.1 bringt Neuerungen für Streaming und maschinelles Lernen
- [Blog] Using Kafka With JUnit
January 3rd, 2017
- [Blog] Monitoring Wikipedia Edit Streams using Apache Flink and Packaging the Application with Dependencies
- [Blog] Managing IoT devices with Kafka and MQTT
- [Blog] The Battle of the Crawlers : Apache Nutch vs StormCrawler
- [Blog] Closure on Apache Nifi
- [Article] 8 data trends on our radar for 2017
- [Blog] Overview: Apache Spark on HDInsight Linux
January 4th, 2017
- [Blog] Asynchronous Processing and Multithreading in Apache Samza, Part I: Design and Architecture
- [Blog] What 2017 Will Bring: 10 More Big Data Predictions. Alex Woodie
- [Blog] Databricks and Apache Spark 2016 Year in Review
- [Presentation] IoT Project Flogo - How to Build an Apache Kafka Connector / Adapter. Kai Wähner Video Slides
- [Blog] Join: Storm in a Teacup, Continued..
- [Blog] Applying Machine Learning to Real Time Streaming Analytics
- [Whiteboard Walkthrough] A Better Way to Build a Fraud Detector: Streaming Data and Microservices Architecture
- [Blog] Kafka Summit 2017 Talk Proposal
- [Conference talk] The Role of Data Virtualization in IoT Integration. Slides Video
January 5th, 2017
- [Blog] Proof of concept using KafkaStreams and KTables
- [Blog] Monitoring Real-Time Uber Data Using Spark Machine Learning, Streaming, and the Kafka API (Part 2)
- [Blog] Log Compaction: Highlights in the Apache Kafka and Stream Processing Community - January 2017. Gwen Shapira
- [Blog] Microservices messaging on Oracle Cloud using Apache Kafka
- [Video] Where Does Apache Geode Fit in CQRS Architectures?
- [Blog] Apache Nifi Installation on Ubuntu
January 6th, 2017
- [Blog] Asynchronous Processing and Multithreading in Apache Samza, Part II: Experiments and Evaluation. Xinyu Liu.
- [Blog] Kafka Avro Scala Example
- [Tutorial] How to use Flume in IOP with Message Hub?
- [Blog] How to Build a Custom Flogo Adapter
January 7th, 2017
- [Meetup] A Deep-dive into Structured Streaming / Predictive Analytics with SparkR. Istanbul Spark Meetup
- [Article] Big Data Processing with Apache Spark - Part 3: Spark Streaming
- [Article] The Impact of Data Grids in IoT
January 8th, 2017
- [Video + Slides] Spring and Big Data. Thomas Risberg
January 9th, 2017
- [Blog] Better Complex Event Processing at Scale Using a Microservices-based Streaming Architecture (Part 1). Mathieu Dumoulin
- [Blog] Release 0.4.0 adds a runner for Apache Apex. Thomas Weise
January 10th, 2017
- [Blog] Real-time Smart City Traffic Monitoring Using Microservices-based Streaming Architecture (Part 2). Mathieu Dumoulin
- [Blog] Google Lauds Outside Influence on Apache Beam, Alex Woodie
- [News] The Apache Software Foundation Announces Apache® Beam™ as a Top-Level Project
- [Blog] Apache Beam graduates to a top-level project
- [Blog] Apache Beam established as a new top-level project, Davor Bonaci
- [News] Google must be Beaming as Apache announces its new top-level projects
- [Blog] Apache Beam graduates to a top-level project, Tyler Akidau. Google Open Source Blog.
- [Blog] Apache Beam graduates from incubation: Try it today on Google Cloud Dataflow, Frances Perry, Google.
- [Blog] What's new in StormCrawler 1.3, Julien Nioche.
January 11th, 2017
- [Meetup] Introduction to Kafka Streams with a Real-Life Example, Apache Kafka DC Slides
- [Webinar] Top 5 IoT Use Cases, Vijay Raja & Dave Shuman
- [Blog] Apache Software Foundation announces two more top-level open source projects. Mike Wheatley
- [Article] Apache Beam unifies batch and streaming for big data, Serdar Yegulalp
- [Slides] Stream Processing as a Foundational Paradigm and Apache Flink's Approach to It. Stephan Ewen.
- [Blog] Kafka vs. MapR Streams: Why MapR? Ian Downard
- [News] Apache Spark 2.1 Improves Structured Streaming, David Ramel.
January 12th, 2017
- [Webinar] January 12: Business insight in minutes with Oracle Stream Analytics
- [Meetup] Processing IoT data with Apache Kafka. Matt Howlett, Confluent. Bay Area Full Stack, Mountain View, CA. Slides
- [Video + Slides] Spring for Apache Kafka, Gary Russel.
- [Blog] Getting Started with Spark Streaming, Python, and Kafka, Robin Moffat.
- [Video + Slides] Architecting for Cloud Native Data: Data Microservices Done Right Using Spring Cloud. Fred Melo
- [Article] Apache Beam and Spark: New coopetition for squashing the Lambda Architecture? Tony Baer, Ovum.
- [Blog] Streaming Analytics for Chain Monitoring, Natalino Busa.
- [Presentation] Staging Reactive Data Pipelines using Kafka as the Backbone. Manchester Geek Nights Video
January 13th, 2017
- [Blog] Apache Kafka: Getting started. G.
- [Blog] Importing JSON into Hadoop via Kafka. By Nuria Ruiz, Andrew Otto, Wikimedia Foundation.
- [Blog] Developing Transactional Microservices Using Aggregates, Event Sourcing and CQRS - Part 2. Part 1
- [Blog] Data Processing and Enrichment in Spark Streaming with Python and Kafka, Robin Moffatt
- [Blog] The Future of Apache Beam, Now a Top-Level Apache Software Foundation Project, Jean-Baptiste Onofre, Talend.
- [Blog] SQL on Apache Apex. Chinmay Kolhatkar
- [Blog] Creating An Email Bot in Apache NiFi, Timothy Spann
January 14th, 2017
- [Slides + Video] Reactive Kafka, Rajini Sivaram. Pivotal
January 15th, 2017
- [Blog] Scaling Kafka with Docker Containers, Jorge Quilcate
January 16th, 2017
- [Presentation] Streaming Real-Time from On-Premise Databases to Big Data in the Amazon Web Services Cloud Slides Video
- [Blog] Updating Materialized Views and Caches Using Kafka. Zach Cox
- [Blog] Ingest Remote Camera Images from Raspberry Pi via MQTT and FTP in Apache NiFi, Timothy Spann.
January 17th, 2017
- [Meetup] Fast Data: Selecting The Right Streaming Technologies For Never Ending Data Sets, Chicago Real-Time Streaming Analytics
- [Meetup] Spark Discussion with Dr. Alex Liu, IBM's Chief Data Scientist. Chicago Spark Users
- [Meetup] Sensor Data Ingestion and Processing with NiFi and Spark. Future of Data, Amsterdam.
- [Blog] Performance Tuning of an Apache Kafka/Spark Streaming System. Mathieu Dumoulin
- [Webinar] Solving the Really Big Tech Problems with IoT
- [Presentation] Building Reactive Fast Data & the Data Lake with Akka, Kafka, Spark, Todd Fritz
January 18th, 2017
- [Meetup] Apache Kafka Meetup with Walmart Labs and Confluent, Apache Kafka Bay Area
- [Meetup] Instrumenting Apache Kafka, Apache Kafka London
- [Meetup] Understanding Big Data Streaming and Apache Flink. Fremont Big Data and Cloud Meetup
- [Meetup] Running Kafka in production. streamprocessing.be Meetup
- [Meetup] TensorFlow & TensorFrames w/ Apache Spark + Deep-dive into Structured Streaming, Apache Spark and more, Milano. Slides
- [Presentation] Reactive integrations with Akka Streams, Konrad Malawski
January 19th, 2017
- [Meetup] gRPC, Kubernetes, Mesos, Spark ML, Structured Streaming, Tensorflow, HDFS, Kafka. Advanced Spark and TensorFlow Meetup, San Francisco, CA
- [Meetup] Fast Data / Event-Driven Architecture with Kafka Streams
- [Webinar] Exploring Reactive Integrations with Akka Streams, Alpakka and Kafka
- [Meetup] Real time product recommendations, Montreal Apache Spark Meetup
- [Presentation] Intro to Big Data AppHub: Demo of HDFS to Kafka and Kafka to HDFS templates. Ashwin Putta, Sanjay Pujare, Devendra Tagare, DataTorrent Slides Video
January 23rd, 2017
January 24th, 2017
- [Meetup] Distributed Reactive Applications & Dynamic load balancing with Akka Streams. Krakow Scala User Group.
January 25th, 2017
January 26th, 2017
- [Webinar] Streaming Data Analytics with Apache Spark Streaming. IBM Analytics
- [Meetup] Kafka Connect & Repeatable deployment of Kafka Streams topologies on kubernetes. Kafka Meetup Utrecht, Netherlands