Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Add sanity check for ulimit -f before attempting curl download #19167

Open
1 task done
fadookie opened this issue Jan 29, 2025 · 3 comments
Open
1 task done

Add sanity check for ulimit -f before attempting curl download #19167

fadookie opened this issue Jan 29, 2025 · 3 comments
Labels
features New features

Comments

@fadookie
Copy link

Verification

Provide a detailed description of the proposed feature

When resolving https://github.com/orgs/Homebrew/discussions/5920 I came to think that the error messaging could be improved when ulimit -f is hit. Right now you get something like this:

eliot@mac ~ % brew doctor 
==> Downloading https://ghcr.io/v2/homebrew/portable-ruby/portable-ruby/blobs/sha256:d9faa506c014dedc0b034a68103ba75c9a58242f4d6c67b6ca0f649c39602bcf
###########################################################################################                                                                                                                    44.6%/opt/homebrew/Library/Homebrew/utils/helpers.sh: line 70: 10650 Filesize limit exceeded: 25   "${HOMEBREW_LIBRARY}/Homebrew/shims/shared/curl" "$@"
Error: Checksum mismatch.
Expected: d9faa506c014dedc0b034a68103ba75c9a58242f4d6c67b6ca0f649c39602bcf
  Actual: fccf4d3edba3426330b3cfc0b967e14df421953da2b57796a34253808ee00647
 Archive: /Users/eliot/Library/Caches/Homebrew/portable-ruby-3.3.7.arm64_big_sur.bottle.tar.gz
To retry an incomplete download, remove the file above.

Way off to the right of the download line there's a cryptic message about Filesize limit exceeded and no suggestion of a possible fix (the fix that is suggested here, deleting the tarball, doesn't work because the file is being truncated when the download is aborted.)

Confounding the problem, it seems on at least some versions of macOS, the default ulimit -f in zsh is quite low (5 MiB), virtually guaranteeing that the tarball for portable-ruby will fail to install during initial setup. I'm on an M4 Pro MacBook Pro 2024 running macOS 15.2 and ran into this issue during initial installation of homebrew.

I'm proposing that a quick sanity check for ulimit -f could be added before attempting to download anything, but especially the initial portable-ruby tarball, and log a warning or error if it's likely too low to allow a successful download.

As long as the server returns a Content-Length header it could be retrieved with a preflight request like so and this could feed into the sanity check:

eliot@mac ~ % curl -sI -H "Authorization: Bearer QQ==" https://ghcr.io/v2/homebrew/portable-ruby/portable-ruby/blobs/sha256:d9faa506c014dedc0b034a68103ba75c9a58242f4d6c67b6ca0f649c39602bcf | awk 'tolower($0) ~ /content-length/ { print $2 }'
11727557

What is the motivation for the feature?

Making it more obvious to the user that they need to raise their ulimit -f when it would prevent a successful file download required by homebrew.

How will the feature be relevant to at least 90% of Homebrew users?

I think it's probably going to become a more commonplace issue. I haven't run into this before on other macs but did on this brand new MacBook running macOS 15.2. It's possible Apple has lowered the default ulimit -f in some recent version of macOS making this more likely, or maybe it's being enforced on curl now and wasn't before. I don't know why I would have run into this otherwise with a brand new machine.

What alternatives to the feature have been considered?

If a code sanity check isn't added it would be good to at least add something prominent to the docs about this.

@MikeMcQuaid
Copy link
Member

I haven't run into this before on other macs but did on this brand new MacBook running macOS 15.2. It's possible Apple has lowered the default ulimit -f in some recent version of macOS making this more likely, or maybe it's being enforced on curl now and wasn't before. I don't know why I would have run into this otherwise with a brand new machine.

$ ulimit -f
unlimited

$ sw_vers
ProductName:		macOS
ProductVersion:		15.2
BuildVersion:		24C101

so don't think it's that. #19177 being on Linux makes me think there's something else going on here.

@fadookie
Copy link
Author

fadookie commented Feb 6, 2025

@MikeMcQuaid Is there anything in particular I can do to help investigate this further?

@MikeMcQuaid
Copy link
Member

@fadookie If you can open a PR with a proposed fix that'd be helpful. Alternatively, if you can produce a series of commands that reproduce this consistently on a default macOS configuration: that'd also help.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
features New features
Projects
None yet
Development

No branches or pull requests

2 participants