Skip to content

Commit

Permalink
feat/docs: document import / export, rename .load / .dump to .import …
Browse files Browse the repository at this point in the history
…/ .export
  • Loading branch information
cablehead committed Jan 16, 2025
1 parent 7226341 commit d9277ea
Show file tree
Hide file tree
Showing 2 changed files with 47 additions and 3 deletions.
45 changes: 45 additions & 0 deletions docs/src/content/docs/reference/import-export.mdx
Original file line number Diff line number Diff line change
@@ -0,0 +1,45 @@
---
title: Import & Export
description: "How to export and import data between cross.stream stores"
sidebar:
order: 5
---

The supervisor exposes two endpoints to facilitate data transfer between stores:

- POST `/import`: Takes JSON frame data and imports it as-is, preserving frame `id` and content `hash`
- POST `/cas`: Stores posted content in CAS and returns its hash

## Commands

`xs.nu` provides two commands to utilize these endpoints:

```nushell
# Export store at $env.XS_ADDR to path
.export <path>
# Import dump at path to $env.XS_ADDR
.import <path>
```

The exported data includes:
- Frame metadata in `frames.jsonl`
- Content files in `cas/` directory

## Version Compatibility

Version 0.1.0 was the first version supporting imports, though the 0.1.0 client can export data from 0.0.9 stores.

## Example

```sh
# Export from remote store
with-env {XS_ADDR: "https://user:[email protected]"} {
.export backup
}

# Import to local store
with-env {XS_ADDR: "./store"} {
.import backup
}
```
5 changes: 2 additions & 3 deletions xs.nu
Original file line number Diff line number Diff line change
Expand Up @@ -86,7 +86,7 @@ export def .remove [id: string] {

export alias .rm = .remove

export def .dump [path: string] {
export def .export [path: string] {
if ($path | path exists) {
print "path exists"
return
Expand All @@ -96,15 +96,14 @@ export def .dump [path: string] {
xs cat (store-addr) | save ($path | path join "frames.jsonl")

open ($path | path join "frames.jsonl") | lines | each {from json | get hash} | uniq | each {|hash|
# if ($hash | is-empty) { return }
let hash_64 = $hash | encode base64
let out_path = $"($path)/cas/($hash_64)"
print $out_path
.cas $hash | save $out_path
}
}

export def .load [path: string] {
export def .import [path: string] {
glob ([$path "cas"] | path join "*") | each {|x|
let want = ($x | path basename | decode base64 | decode)
let got = cat $x | xs cas-post (store-addr)
Expand Down

0 comments on commit d9277ea

Please sign in to comment.