Merge pull request #35 from imaifactory/feature-disablling-ensure-con…

…nection

Optionalize ensuring stream connection

README.md

@@ -119,6 +119,10 @@ It should be in form like "us-east-1", "us-west-2".
 Refer to [Regions and Endpoints in AWS General Reference](http://docs.aws.amazon.com/general/latest/gr/rande.html#ak_region)
 for supported regions.
 
+### ensure_stream_connection
+
+When enabled, the plugin checks and ensures a connection to the stream you are using by [DescribeStream](http://docs.aws.amazon.com/kinesis/latest/APIReference/API_DescribeStream.html) and throws exception if it fails. Enabled by default.
+
 ### http_proxy
 
 Proxy server, if any.
@@ -196,7 +200,7 @@ Integer, default is 3. When **order_events** is false, the plugin will put multi
 records to Amazon Kinesis in batches using PutRecords. A set of records in a batch
 may fail for reasons documented in the Kinesis Service API Reference for PutRecords.
 Failed records will be retried **retries_on_putrecords** times. If a record
-fails all retries an error log will be emitted. 
+fails all retries an error log will be emitted.
 
 ### use_yajl
 

lib/fluent/plugin/out_kinesis.rb

@@ -41,6 +41,7 @@ class OutputFilter < Fluent::BufferedOutput
     # The 'region' parameter is optional because
     # it may be set as an environment variable.
     config_param :region,      :string, default: nil
+    config_param :ensure_stream_connection, :bool, default: true
 
     config_param :profile,          :string, :default => nil
     config_param :credentials_path, :string, :default => nil
@@ -102,7 +103,9 @@ def start
       detach_multi_process do
         super
         load_client
-        check_connection_to_stream
+        if @ensure_stream_connection
+          check_connection_to_stream
+        end
       end
     end
 

test/plugin/test_out_kinesis.rb

@@ -143,6 +143,7 @@ def test_configure_with_more_options
     conf = %[
       stream_name test_stream
       region us-east-1
+      ensure_stream_connection false
       http_proxy http://proxy:3333/
       partition_key test_partition_key
       partition_key_expr record
@@ -154,6 +155,7 @@ def test_configure_with_more_options
     d = create_driver(conf)
     assert_equal 'test_stream', d.instance.stream_name
     assert_equal 'us-east-1', d.instance.region
+    assert_equal false, d.instance.ensure_stream_connection
     assert_equal 'http://proxy:3333/', d.instance.http_proxy
     assert_equal 'test_partition_key', d.instance.partition_key
     assert_equal 'Proc',