Make the wildfire connector great again

[ghost]

Can we improve our connection retry logic for the connector? I have the following on a python-cb-wildfire-connector-2.5-6.x86_64 version:

2017-08-05 18:15:06,626: init: INFO: Received a network connection error from https:// : HTTPSConnectionPool(host=' ', port= ): Max retries exceeded with url: /api/info (Caused by NewConnectionError('<urllib3.connection.VerifiedHTTPSConnection object at 0x7f1362d72150>: Failed to establish a new connection: [Errno 111] Connection refused',))
2017-08-05 18:15:06,626: init: INFO: Failed to connect to Cb Response Server, retrying in 30 secs...
2017-08-05 18:15:51,676: init: INFO: Received a network connection error from https:// : HTTPSConnectionPool(host=' ' ', port= ): Max retries exceeded with url: /api/info (Caused by NewConnectionError('<urllib3.connection.VerifiedHTTPSConnection object at 0x7f1362d72a90>: Failed to establish a new connection: [Errno 111] Connection refused',))
2017-08-05 18:15:51,676: init: INFO: Failed to connect to Cb Response Server, retrying in 30 secs...
2017-08-05 18:16:36,726: init: INFO: Received a network connection error from https:// : HTTPSConnectionPool(host=' ', port= ): Max retries exceeded with url: /api/info (Caused by NewConnectionError('<urllib3.connection.VerifiedHTTPSConnection object at 0x7f1362d72bd0>: Failed to establish a new connection: [Errno 111] Connection refused',))
2017-08-05 18:16:36,726: init: INFO: Failed to connect to Cb Response Server, retrying in 30 secs...
2017-08-05 18:17:06,757: daemon: CRITICAL: Could not connect to Cb server at https:// (ConfigurationError)
2017-08-05 18:17:06,757: daemon: CRITICAL: Traceback: Traceback (most recent call last):
File "site-packages/cbint/utils/daemon.py", line 166, in start
File "cbopensource/connectors/wildfire/bridge.py", line 182, in validate_config
File "site-packages/cbint/utils/detonation/init.py", line 172, in validate_config
ConfigurationError: Could not connect to Cb server at https://

This was during a start of services:

2017-08-05 18:22:02 [7920] ---- Starting cb-enterprise(6.1.2.170707.2323) services.... ----
2017-08-05 18:22:08 [7920] Started cb-supervisord (PID 8378)
2017-08-05 18:28:11 [7920] ---- cb-enterprise successfully started ----

Would be great if the connector didn't terminate after 2 minutes of connectivity problems, especially since instances with larger datastores in 6.1 will now take several minutes to completely build the solr index and start all services.

[jgarman]

I like your style @adamnadrowski . We'll make it so the daemon never gives up trying to connect to the Cb Response server.