-
Notifications
You must be signed in to change notification settings - Fork 1.3k
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Scalable id storage #3292
Comments
I'd suggest splitting the root directory by the first letter of the path, as you suggested. It's a pretty common pattern in file-based storage systems (I remember that e.g., MediaWiki uses it for image storage), so it shouldn't be too confusing to people... The current layout does slow down the GH UI considerably. My VS Code still holds up, but it's hard to navigate such a long list of directories. |
I suggest also changing the title of this issue, to "toward more scalable perma-id storage", as we're not yet discussing implementing a fully scalable solution (i.e., database), just a stop-gap (i.e., manual directory sharding). |
Need to consider the case handling of some file systems. How confusing would it be to put all I think it would be fun to use hex sharding! Would directories like |
|
The current number of top-level ids is ... 1000+? It can be a bit difficult to navigate and manage in the GitHub UI and console. We may need to explore solutions to this. At some point a database would be needed, but let's not go there yet. A short term solution would be to shard the ids into subdirs. A top-level .htaccess file could likely be constructed that would internally redirect to a subdir based on the first path letter. That would likely work and cut down max ids per dir significantly. But it could cause user confusion.
The text was updated successfully, but these errors were encountered: