Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

PDB Serialization Pt. 4, add reexport-pdb command #103

Draft
wants to merge 48 commits into
base: main
Choose a base branch
from
Draft
Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
48 commits
Select commit Hold shift + click to select a range
f8fec0c
feat(pdb): Implement support for serialization of table pages
Holzhaus Apr 22, 2022
4717a0d
fix(pdb): adhere to rows alignment to type alignment when writing
Swiftb0y Sep 29, 2022
8f8fe48
WIP document some debugging/RE-ing results related to pages
Swiftb0y Sep 7, 2022
33931da
add CLI command reexport-pdb to test serialization
erictapen Oct 12, 2023
846cde0
init nix flake for development environment
erictapen Nov 11, 2023
77db5c2
wip serialize pdb
erictapen Nov 11, 2023
ebefd8d
nixpkgs-fmt
erictapen Nov 11, 2023
f817a3d
fix missing page_size problem when writing Page
erictapen Nov 11, 2023
1f303ef
define placeholders for RowType to not run into row alignment issues
erictapen Nov 11, 2023
90e8727
don't write RowGroup's again, as they are already written by Page
erictapen Nov 11, 2023
0498640
add rehex to devShell
erictapen Nov 11, 2023
d7df78e
fix import warning
erictapen Nov 11, 2023
6d42449
write paddings after file header + add few dbg! and println! lines
summersamara Nov 12, 2023
175dfac
make PageIndex content public
erictapen Nov 12, 2023
29f36d6
write pages after sorting them per page_index
summersamara Nov 12, 2023
4879e56
consume Result
erictapen Nov 12, 2023
ca12775
add TODO comment to Page write_options methods
summersamara Nov 12, 2023
a645145
small comment about order of Row offsets
erictapen Nov 12, 2023
3f6e975
fix row_offset so that they starts at 0 after page header
summersamara Nov 13, 2023
4960642
todo for num_rows_large
erictapen Nov 13, 2023
7efdb7f
comment out align_by call when calculating row_offset
summersamara Nov 13, 2023
941c6b1
rowgroup: don't align at all for now
erictapen Nov 13, 2023
5d21446
dont restor position after read/writing Artist name
summersamara Nov 13, 2023
9f3ec4e
reverse the rowgroups before writing so we write complete rowgroups f…
erictapen Nov 13, 2023
871285f
debug comments
erictapen Nov 13, 2023
e2fde66
write Album name in a similar fashion to the Artist name
erictapen Nov 13, 2023
1b514fc
make calculate_name_seek way more pretty (and three times longer)
erictapen Nov 13, 2023
6d1700a
fix some warnings
erictapen Nov 13, 2023
213a64e
run pre-commit checks, install necessary dependencies
erictapen Nov 13, 2023
8d8f0ed
(fix): demo_tracks_header test
erictapen Nov 25, 2023
a128867
- Reverse rows when writing instead of reversing row_groups
summersamara Dec 24, 2023
eba9d81
chore(changelog): Update changelog after merge
Holzhaus Aug 13, 2023
8a0bcf7
(refactor): update binrw to 0.13
erictapen Dec 26, 2023
f84d8b9
(refactor): update binrw to 0.13
erictapen Nov 12, 2023
e7d9164
(refactor): remove unnecessary Clone trait
erictapen Nov 15, 2023
6c46bf5
(refactor): Add comment about expensive clone
erictapen Nov 23, 2023
cb18fa4
chore(pdb): Remove trailing whitespace in comment
Holzhaus Nov 30, 2023
bf455f3
chore(anlz): Fix typo in comment
Holzhaus Nov 30, 2023
a04d663
ci(pre-commit): Update pre-commit hooks
Holzhaus Nov 30, 2023
ac68962
chore(release): Release version v0.2.1
Holzhaus Nov 30, 2023
414656c
wip fix stuff for binrw update
erictapen Dec 26, 2023
56cfd1a
fix build for now...
erictapen Dec 27, 2023
aeab75e
Merge remote-tracking branch 'origin/main' into reexport-pdb
erictapen Dec 27, 2023
f3325f2
rexport-pdb feature to reexport a (more) valid pdb file
summersamara Dec 27, 2023
19e3256
Add comments to Page write_options
summersamara Dec 28, 2023
b0e2441
formatting
erictapen Dec 27, 2023
6c84417
fix: tests compile again
erictapen Dec 28, 2023
fffea63
remove flake.nix file as it's not upstreamable
erictapen Feb 26, 2024
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
79 changes: 79 additions & 0 deletions src/main.rs
Original file line number Diff line number Diff line change
Expand Up @@ -41,6 +41,15 @@ enum Commands {
#[arg(value_name = "PDB_FILE")]
path: PathBuf,
},
/// Read a Pioneer Database (`.PDB`) file and write the serialization to a different place.
ReexportPDB {
/// File to parse.
#[arg(value_name = "PDB_IN_FILE")]
inpath: PathBuf,
/// File to write.
#[arg(value_name = "PDB_OUT_FILE")]
outpath: PathBuf,
},
/// Parse and dump a Pioneer Settings (`*SETTING.DAT`) file.
DumpSetting {
/// File to parse.
Expand Down Expand Up @@ -150,6 +159,75 @@ fn dump_pdb(path: &PathBuf) -> rekordcrate::Result<()> {
Ok(())
}

fn reexport_pdb(inpath: &PathBuf, outpath: &PathBuf) -> rekordcrate::Result<()> {
use binrw::BinWrite;
use rekordcrate::pdb::PageIndex;
use std::collections::HashMap;
use std::io::Seek;

let mut reader = std::fs::File::open(inpath)?;
let header = Header::read(&mut reader)?;

println!("Header {:?}", header);

let mut writer = std::fs::File::create(outpath)?;

let endian = binrw::Endian::NATIVE;
header.write_options(&mut writer, endian, ())?;

let writer_offset = writer.stream_position().map_err(binrw::Error::Io)?;

let header_padding: usize = (header.page_size - writer_offset as u32)
.try_into()
.unwrap();

vec![0u8; header_padding].write_options(&mut writer, endian, ())?;

let mut pages_hash_map = HashMap::new();
let mut max_page_index = 0;
for (_, table) in header.tables.iter().enumerate() {
for page in header
.read_pages(
&mut reader,
binrw::Endian::NATIVE,
(&table.first_page, &table.last_page),
)
.unwrap()
.into_iter()
{
println!(" {:?}", page);
let PageIndex(index) = page.page_index;

if index > max_page_index {
max_page_index = index;
}

pages_hash_map.insert(index, page);
}
}

for i in 1..(max_page_index + 1) {
if let Some(page) = pages_hash_map.get(&i) {
let PageIndex(next_index) = page.next_page;
let next_page_num_rows = if let Some(next_page) = pages_hash_map.get(&next_index) {
next_page.num_rows()
} else {
0
};

page.write_options(
&mut writer,
endian,
(header.page_size, next_page_num_rows as u32),
)?;
} else {
vec![0u8; header.page_size as usize].write_options(&mut writer, endian, ())?;
}
}

Ok(())
}

fn dump_setting(path: &PathBuf) -> rekordcrate::Result<()> {
let mut reader = std::fs::File::open(path)?;
let setting = Setting::read(&mut reader)?;
Expand All @@ -165,6 +243,7 @@ fn main() -> rekordcrate::Result<()> {
match &cli.command {
Commands::ListPlaylists { path } => list_playlists(path),
Commands::DumpPDB { path } => dump_pdb(path),
Commands::ReexportPDB { inpath, outpath } => reexport_pdb(inpath, outpath),
Commands::DumpANLZ { path } => dump_anlz(path),
Commands::DumpSetting { path } => dump_setting(path),
}
Expand Down
Loading
Loading