Broken pipe error with upload-source

[chintanp]

Similar to #77 , I get the broken pipe error with the following command:

tilesets upload-source <user-name> <project-name> populated_places.geojson.ld with the environment variable and the --token flag. The json is from the example on the website, and I tried another geojson file that passes the validation test. Also, tried with --no-validation flag with the same result.

The error is below:

Traceback (most recent call last):
  File "/home/chintan/anaconda3/lib/python3.8/site-packages/urllib3/connectionpool.py", line 670, in urlopen
    httplib_response = self._make_request(
  File "/home/chintan/anaconda3/lib/python3.8/site-packages/urllib3/connectionpool.py", line 392, in _make_request
    conn.request(method, url, **httplib_request_kw)
  File "/home/chintan/anaconda3/lib/python3.8/http/client.py", line 1240, in request
    self._send_request(method, url, body, headers, encode_chunked)
  File "/home/chintan/anaconda3/lib/python3.8/http/client.py", line 1286, in _send_request
    self.endheaders(body, encode_chunked=encode_chunked)
  File "/home/chintan/anaconda3/lib/python3.8/http/client.py", line 1235, in endheaders
    self._send_output(message_body, encode_chunked=encode_chunked)
  File "/home/chintan/anaconda3/lib/python3.8/http/client.py", line 1045, in _send_output
    self.send(chunk)
  File "/home/chintan/anaconda3/lib/python3.8/http/client.py", line 967, in send
    self.sock.sendall(data)
  File "/home/chintan/anaconda3/lib/python3.8/ssl.py", line 1204, in sendall
    v = self.send(byte_view[count:])
  File "/home/chintan/anaconda3/lib/python3.8/ssl.py", line 1173, in send
    return self._sslobj.write(data)
BrokenPipeError: [Errno 32] Broken pipe

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "/home/chintan/anaconda3/lib/python3.8/site-packages/requests/adapters.py", line 439, in send
    resp = conn.urlopen(
  File "/home/chintan/anaconda3/lib/python3.8/site-packages/urllib3/connectionpool.py", line 724, in urlopen
    retries = retries.increment(
  File "/home/chintan/anaconda3/lib/python3.8/site-packages/urllib3/util/retry.py", line 403, in increment
    raise six.reraise(type(error), error, _stacktrace)
  File "/home/chintan/anaconda3/lib/python3.8/site-packages/urllib3/packages/six.py", line 734, in reraise
    raise value.with_traceback(tb)
  File "/home/chintan/anaconda3/lib/python3.8/site-packages/urllib3/connectionpool.py", line 670, in urlopen
    httplib_response = self._make_request(
  File "/home/chintan/anaconda3/lib/python3.8/site-packages/urllib3/connectionpool.py", line 392, in _make_request
    conn.request(method, url, **httplib_request_kw)
  File "/home/chintan/anaconda3/lib/python3.8/http/client.py", line 1240, in request
    self._send_request(method, url, body, headers, encode_chunked)
  File "/home/chintan/anaconda3/lib/python3.8/http/client.py", line 1286, in _send_request
    self.endheaders(body, encode_chunked=encode_chunked)
  File "/home/chintan/anaconda3/lib/python3.8/http/client.py", line 1235, in endheaders
    self._send_output(message_body, encode_chunked=encode_chunked)
  File "/home/chintan/anaconda3/lib/python3.8/http/client.py", line 1045, in _send_output
    self.send(chunk)
  File "/home/chintan/anaconda3/lib/python3.8/http/client.py", line 967, in send
    self.sock.sendall(data)
  File "/home/chintan/anaconda3/lib/python3.8/ssl.py", line 1204, in sendall
    v = self.send(byte_view[count:])
  File "/home/chintan/anaconda3/lib/python3.8/ssl.py", line 1173, in send
    return self._sslobj.write(data)
urllib3.exceptions.ProtocolError: ('Connection aborted.', BrokenPipeError(32, 'Broken pipe'))

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "/home/chintan/anaconda3/bin/tilesets", line 8, in <module>
    sys.exit(cli())
  File "/home/chintan/anaconda3/lib/python3.8/site-packages/click/core.py", line 829, in __call__
    return self.main(*args, **kwargs)
  File "/home/chintan/anaconda3/lib/python3.8/site-packages/click/core.py", line 782, in main
    rv = self.invoke(ctx)
  File "/home/chintan/anaconda3/lib/python3.8/site-packages/click/core.py", line 1259, in invoke
    return _process_result(sub_ctx.command.invoke(sub_ctx))
  File "/home/chintan/anaconda3/lib/python3.8/site-packages/click/core.py", line 1066, in invoke
    return ctx.invoke(self.callback, **ctx.params)
  File "/home/chintan/anaconda3/lib/python3.8/site-packages/click/core.py", line 610, in invoke
    return callback(*args, **kwargs)
  File "/home/chintan/anaconda3/lib/python3.8/site-packages/click/decorators.py", line 21, in new_func
    return f(get_current_context(), *args, **kwargs)
  File "/home/chintan/anaconda3/lib/python3.8/site-packages/mapbox_tilesets/scripts/cli.py", line 516, in upload_source
    return _upload_source(
  File "/home/chintan/anaconda3/lib/python3.8/site-packages/mapbox_tilesets/scripts/cli.py", line 592, in _upload_source
    resp = getattr(s, method)(
  File "/home/chintan/anaconda3/lib/python3.8/site-packages/requests/sessions.py", line 578, in post
    return self.request('POST', url, data=data, json=json, **kwargs)
  File "/home/chintan/anaconda3/lib/python3.8/site-packages/requests/sessions.py", line 530, in request
    resp = self.send(prep, **send_kwargs)
  File "/home/chintan/anaconda3/lib/python3.8/site-packages/requests/sessions.py", line 643, in send
    r = adapter.send(request, **kwargs)
  File "/home/chintan/anaconda3/lib/python3.8/site-packages/requests/adapters.py", line 498, in send
    raise ConnectionError(err, request=request)
requests.exceptions.ConnectionError: ('Connection aborted.', BrokenPipeError(32, 'Broken pipe'))

My OS info:
lsb_release -a

Distributor ID:	Ubuntu
Description:	Ubuntu 20.04.1 LTS
Release:	20.04
Codename:	focal

Mapbox-tilesets info:
pip show mapbox-tilesets

Name: mapbox-tilesets
Version: 1.4.2
Summary: CLI for interacting with and preparing data for the Tilesets API
Home-page: https://github.com/mapbox/tilesets-cli
Author: Mapbox
Author-email: [email protected]
License: BSD-2
Location: /home/chintan/anaconda3/lib/python3.8/site-packages
Requires: jsonseq, requests-toolbelt, boto3, requests, jsonschema, click, cligj
Required-by: 

Any help in this regard is appreciated?

[kent-williams]

@chintanp I was experiencing this same issue until I realized the API token I was using didn't have the right access. I created a new one with all tileset and upload permissions and it worked fine.

[octopusabhiraj]

I am facing the same issue. Same mapbox-tilesets info as @chintanp. I tried @kent-williams's method by generating a new token. It did not help.

pip show mapbox-tilesets

Name: mapbox-tilesets
Version: 1.4.2
Summary: CLI for interacting with and preparing data for the Tilesets API
Home-page: https://github.com/mapbox/tilesets-cli
Author: Mapbox
Author-email: [email protected]
License: BSD-2
Location: /Users/abhirajdeshpande/Documents/venvs/3.7Env/lib/python3.7/site-packages
Requires: jsonschema, boto3, requests, cligj, requests-toolbelt, click, jsonseq
Required-by:

[chintanp]

@octopusabhiraj Not a solution to the above problem exactly, but if you want to upload tilesets to Mapbox, then I was able to achieve the same using a round-about way, which I describe here: https://evi-dss.readthedocs.io/en/latest/evi_des.html#plotting

Let me know if you have any questions about the above method.

[octopusabhiraj]

@chintanp converting to .mbtiles is an efficient way to save space and upload time. The problem is that I want to programmatically upload the tileset to MapBox and I don't think there are any APIs to do so. Please let me know if there are any. I cannot upload the tileset manually to MapBox Studio. Thanks!

[geohacker]

Hello! Just wanted to say that I'm running into this as well but I confirmed it's not the token, because it works for a 377M file but not for a 607M file. Could it be because the file is too big? I'm running tilesets-cli v1.5.1 on a python:3.6.11-buster docker container.

Traceback (most recent call last):
  File "/usr/local/lib/python3.6/site-packages/urllib3/connectionpool.py", line 600, in urlopen
    chunked=chunked)
  File "/usr/local/lib/python3.6/site-packages/urllib3/connectionpool.py", line 354, in _make_request
    conn.request(method, url, **httplib_request_kw)
  File "/usr/local/lib/python3.6/http/client.py", line 1272, in request
    self._send_request(method, url, body, headers, encode_chunked)
  File "/usr/local/lib/python3.6/http/client.py", line 1318, in _send_request
    self.endheaders(body, encode_chunked=encode_chunked)
  File "/usr/local/lib/python3.6/http/client.py", line 1267, in endheaders
    self._send_output(message_body, encode_chunked=encode_chunked)
  File "/usr/local/lib/python3.6/http/client.py", line 1077, in _send_output
    self.send(chunk)
  File "/usr/local/lib/python3.6/http/client.py", line 998, in send
    self.sock.sendall(data)
  File "/usr/local/lib/python3.6/ssl.py", line 975, in sendall
    v = self.send(byte_view[count:])
  File "/usr/local/lib/python3.6/ssl.py", line 944, in send
    return self._sslobj.write(data)
  File "/usr/local/lib/python3.6/ssl.py", line 642, in write
    return self._sslobj.write(data)
BrokenPipeError: [Errno 32] Broken pipe

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "/usr/local/lib/python3.6/site-packages/requests/adapters.py", line 449, in send
    timeout=timeout
  File "/usr/local/lib/python3.6/site-packages/urllib3/connectionpool.py", line 638, in urlopen
    _stacktrace=sys.exc_info()[2])
  File "/usr/local/lib/python3.6/site-packages/urllib3/util/retry.py", line 368, in increment
    raise six.reraise(type(error), error, _stacktrace)
  File "/usr/local/lib/python3.6/site-packages/urllib3/packages/six.py", line 685, in reraise
    raise value.with_traceback(tb)
  File "/usr/local/lib/python3.6/site-packages/urllib3/connectionpool.py", line 600, in urlopen
    chunked=chunked)
  File "/usr/local/lib/python3.6/site-packages/urllib3/connectionpool.py", line 354, in _make_request
    conn.request(method, url, **httplib_request_kw)
  File "/usr/local/lib/python3.6/http/client.py", line 1272, in request
    self._send_request(method, url, body, headers, encode_chunked)
  File "/usr/local/lib/python3.6/http/client.py", line 1318, in _send_request
    self.endheaders(body, encode_chunked=encode_chunked)
  File "/usr/local/lib/python3.6/http/client.py", line 1267, in endheaders
    self._send_output(message_body, encode_chunked=encode_chunked)
  File "/usr/local/lib/python3.6/http/client.py", line 1077, in _send_output
    self.send(chunk)
  File "/usr/local/lib/python3.6/http/client.py", line 998, in send
    self.sock.sendall(data)
  File "/usr/local/lib/python3.6/ssl.py", line 975, in sendall
    v = self.send(byte_view[count:])
  File "/usr/local/lib/python3.6/ssl.py", line 944, in send
    return self._sslobj.write(data)
  File "/usr/local/lib/python3.6/ssl.py", line 642, in write
    return self._sslobj.write(data)
urllib3.exceptions.ProtocolError: ('Connection aborted.', BrokenPipeError(32, 'Broken pipe'))

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "/usr/local/bin/tilesets", line 8, in <module>
    sys.exit(cli())
  File "/usr/local/lib/python3.6/site-packages/click/core.py", line 829, in __call__
    return self.main(*args, **kwargs)
  File "/usr/local/lib/python3.6/site-packages/click/core.py", line 782, in main
    rv = self.invoke(ctx)
  File "/usr/local/lib/python3.6/site-packages/click/core.py", line 1259, in invoke
    return _process_result(sub_ctx.command.invoke(sub_ctx))
  File "/usr/local/lib/python3.6/site-packages/click/core.py", line 1066, in invoke
    return ctx.invoke(self.callback, **ctx.params)
  File "/usr/local/lib/python3.6/site-packages/click/core.py", line 610, in invoke
    return callback(*args, **kwargs)
  File "/usr/local/lib/python3.6/site-packages/click/decorators.py", line 21, in new_func
    return f(get_current_context(), *args, **kwargs)
  File "/usr/local/lib/python3.6/site-packages/mapbox_tilesets/scripts/cli.py", line 519, in upload_source
    ctx, username, id, features, no_validation, quiet, replace, token, indent
  File "/usr/local/lib/python3.6/site-packages/mapbox_tilesets/scripts/cli.py", line 599, in _upload_source
    "Content-type": monitor.content_type,
  File "/usr/local/lib/python3.6/site-packages/requests/sessions.py", line 593, in put
    return self.request('PUT', url, data=data, **kwargs)
  File "/usr/local/lib/python3.6/site-packages/requests/sessions.py", line 533, in request
    resp = self.send(prep, **send_kwargs)
  File "/usr/local/lib/python3.6/site-packages/requests/sessions.py", line 646, in send
    r = adapter.send(request, **kwargs)
  File "/usr/local/lib/python3.6/site-packages/requests/adapters.py", line 498, in send
    raise ConnectionError(err, request=request)
requests.exceptions.ConnectionError: ('Connection aborted.', BrokenPipeError(32, 'Broken pipe'))

[mhuconcern]

Experiencing the same thing as @geohacker not working for certain files, but mine is of the magnitude of 500kb (succeeded) vs 2.3mb (did not succeeded) vs 122.6MB (succeeded) All three files are similar in geometry type and i used same function to create them, i was able to successfully create mbtiles from the three of them

[geohacker]

@mhuconcern I was able to speak to someone at Mapbox and they found that reducing the coordinate precision of my GeoJSON fixed my issue. I'm now using a GeoJSON that has a precision upto 5 digits.

[mhuconcern]

Mine is fixed by changing the id of the tileset, one of my files had a long name. The upload-source doesn't throw good error messages

[thex1le]

Mine is fixed by changing the id of the tileset, one of my files had a long name. The upload-source doesn't throw good error messages

I had the exact same error uploading a 3.6 gig file, when I reduced the len of the ID then it uploaded correctly. This is cleary is a bug where they should check for the length of an ID and throw a better error.