com.netflix.turbine.streaming

Class TurbineStreamingConnection<T extends TurbineData>

java.lang.Object

com.netflix.turbine.streaming.TurbineStreamingConnection<T>

Type Parameters:

T -

All Implemented Interfaces:

TurbineDataHandler<T>

public class TurbineStreamingConnection<T extends TurbineData>
extends java.lang.Object
implements TurbineDataHandler<T>

Class that represents an output stream from a ClusterMonitor

TurbineStreamingConnection provides the following functionality

• Streaming filtered data that matches a given data type
• Streaming filtered data that matches a given data name
• Streaming filtered data that starts with a given name prefix
• Sorting data according to the specified RelevanceConfig
• Buffering data for a specified string delay
• De-duping updates on data that is not changing often
• Sending ping heartbeats to signal inactivity to downstream StreamingDataHandler listeners

Nested Class Summary

Nested Classes

Modifier and Type	Class and Description
static class	TurbineStreamingConnection.UnitTest

Field Summary

Fields

Modifier and Type	Field and Description
protected java.util.Set<java.lang.String>	filterPrefixes
protected java.lang.String	name
protected StreamingDataHandler	streamHandler
protected java.util.concurrent.ConcurrentHashMap<java.lang.String,java.lang.Object>	streamingConnectionSession Signal controls used to co-ordinate the data handler thread and the ping thread.
protected int	streamingDelay

Constructor Summary

Constructors

Constructor and Description
TurbineStreamingConnection(StreamingDataHandler sHandler, java.util.Collection<TurbineStreamServlet.FilterCriteria> criteria, int delay)

Method Summary

Methods

Modifier and Type	Method and Description
PerformanceCriteria	getCriteria() Indicate how big a buffer and how many threads are required to dispatch data reliably to this handler
java.lang.String	getName() Name for identification purposes
void	handleData(java.util.Collection<T> data) Receive a collection of data from an turbine data monitor
void	handleHostLost(Instance host) Notification that a data monitor has gone away or stopped monitoring.
void	waitOnConnection() This will continue streaming responses and will block until the client connection breaks or the cluster monitor above fails to give data
protected void	writeToStream(java.util.Collection<? extends TurbineData> dataCollection)

Methods inherited from class java.lang.Object

clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait

Field Detail

name

protected final java.lang.String name

streamHandler

protected final StreamingDataHandler streamHandler

streamingDelay

protected int streamingDelay

filterPrefixes

protected final java.util.Set<java.lang.String> filterPrefixes

streamingConnectionSession

protected java.util.concurrent.ConcurrentHashMap<java.lang.String,java.lang.Object> streamingConnectionSession

Signal controls used to co-ordinate the data handler thread and the ping thread. In case the relevance criteria is specified, we need to periodically sort the data using the ping thread. But the ping thread runs once every 3 seconds and hence the sorted results (relevant data) can take a while to surface. Furthermore, we should sort only when we have enough relevant data, and this comes from another thread - the data handler thread(s) Also, the data handler thread should try as much as possible to not send non-relevant data out to the wire, but what is relevant is decided by the other 'ping' thread. For balancing a fast real-time experience with good relevant (sorted topN) data, we need to co-ordinate these 2 independent threads without any expensive synchronization primitives. Hence I'm using the following mechanism based on a state machine model to achieve this. The different states are DoNotSort --> StartSorting --> Sorted and occur in the order specified. The state transitions occur exactly in this direction and there is no other transition. 1. DoNotSort - is the initial state that we start out in where the sorting thread ( calling waitOnConnection() ) is not allowed to sort, since there isn't enough data yet. The data thread is still gathering enough data and determining whether it is worthy of being sorted. While in this state the sorting thread spins for a max of 3 seconds for each relevant metric type, and the data thread does not spit out any data to downstream listeners. 2. StartSorting - once the data thread gets enough data, it signals the sorting thread to start doing it's work. Again during this state, the data thread does not spit any data out. 3. Sorted - This is the last state and is signalled by the sorting thread, which then unblocks the data thread who can then stream data out as long as the data is within the topN list.

Constructor Detail

TurbineStreamingConnection

public TurbineStreamingConnection(StreamingDataHandler sHandler,
                          java.util.Collection<TurbineStreamServlet.FilterCriteria> criteria,
                          int delay)
                           throws java.lang.Exception

Throws:

java.lang.Exception

Method Detail

getName

public java.lang.String getName()

Description copied from interface: TurbineDataHandler

Name for identification purposes

Specified by:

getName in interface TurbineDataHandler<T extends TurbineData>

Returns:

String

waitOnConnection

public void waitOnConnection()

This will continue streaming responses and will block until the client connection breaks or the cluster monitor above fails to give data

handleData

public void handleData(java.util.Collection<T> data)

Description copied from interface: TurbineDataHandler

Receive a collection of data from an turbine data monitor

Specified by:

handleData in interface TurbineDataHandler<T extends TurbineData>

handleHostLost

public void handleHostLost(Instance host)

Description copied from interface: TurbineDataHandler

Notification that a data monitor has gone away or stopped monitoring.

e.g when an InstanceMonitor shuts down due to a connection reset or when a aggregator cluster monitor has been stopped.

Specified by:

handleHostLost in interface TurbineDataHandler<T extends TurbineData>

writeToStream

protected void writeToStream(java.util.Collection<? extends TurbineData> dataCollection)

getCriteria

public PerformanceCriteria getCriteria()

Description copied from interface: TurbineDataHandler

Indicate how big a buffer and how many threads are required to dispatch data reliably to this handler

Specified by:

getCriteria in interface TurbineDataHandler<T extends TurbineData>

Returns:

PerformanceCriteria