Board request: NanoPc CM3588 NAS #142
Comments
|
I too would like to know how support for the CM3588 would be added. |
|
You can use one of the existing platforms as a template. Board-specific code resides in the .DSC file and RockchipPlatformLib. The DTB should indeed be imported from Armbian's rk 6.1 branch. Right now it's only used for booting Linux, UEFI itself does not care about it. By the way, the PCIe code does not currently support bifurcation (which the CM3588 appears to be using), so things like NVME drives won't be usable in UEFI. |
|
Thank you for the response.
What exactly would be needed for PCIe bifurcation support? I ask because my primary use case for the board is as a NAS, with... you guessed it, NVMe drives. EDIT: OTOH, if it's only in UEFI where bifurcation doesn't work then perhaps it's not that much of a problem for the use case of a NAS? |
|
Bifurcation missing in UEFI means you can't boot from NVME. Maybe one of the slots would work at x1 link depending on how things are wired, but I don't know for sure. It shouldn't be difficult to implement. It mostly needs changes to the PCIe 3 PHY driver and some nice user-configurable settings for lane mapping. I haven't had the hardware to test this and hence no interest really, but I'm going to accept pull requests if anyone's interested in taking on this task. If your use-case is booting Linux with a device tree off SD/eMMC/USB, then you probably don't care about any of the above. |
|
I have begun simple work to get it ported to the device here. The fork/branch produces a bootable build of EDK2. This device is almost identical to the T6 which I used as the base. I don't particularly require NVMe boot however the first NVMe device did show as a boot option. I'm relatively new to the ARM scene and not very adept at reading device schematics however if I can reverse engineer the values from the T6 to understand how this is mapped I should be able to clean this up and submit a PR. @mariobalanica Maybe you can give me a simple example of how this mapping done? Otherwise, if anyone want's to take a crack at it you can build from my fork |
|
I tried your fork @navarro967, but my keyboard wouldn't get detected. If I was lucky I could get into the boot options by pressing Escape, but once I got in I couldn't navigate anything. Were you able to get this working on your board, or are you experiencing the same issue? |
|
I just got my hardware. |
@jjpe |
Not even if a wrong image is flashed, which at least in theory, could cause electrical damage by driving e.g. some I/O ports incorrectly? If that's true, then that is very useful to know indeed. |
Software wise they can't be bricked just like a regular pc, ofc physical damage can be done i.e plugging stuff where they don't belong but that rarely happens |
I was able to, I plugged my Keychron K10 into a USB A port and was able to enter the BIOS and make changes. |
|
I have tried 5 different Keyboards, and the only one that worked was a Microsoft Sculpt (notably, the only one which uses a wireless dongle). Every one of them worked, once booted into the OS. I couldn't find any error messages, the only thing i noticed was that all keyboards get power when plugged in, but only for a second before turning off again. |
|
Thanks for kicking this off @navarro967! I rebased your fork on the latest master to bring in the PCIe bifurcation support, and I can confirm that the firmware loads and sees my NVMe drive. I haven't actually managed to boot into an OS yet. I have a working Ubuntu installation on the eMMC -- I know it's working because if I remove the sd card with edk2 then it boots -- but selecting it in the boot configuration menu just flashes the screen black and brings me back to edk2. If anyone has suggestions about how to debug it, I'd be grateful. Otherwise, I'm going to poke around with GPIO settings. |
|
Got my NAS board today and tried @navarro967's fork, flashed to eMMC via the FriendlyWrt web flasher. Console is serial, connected to the three pin header. Looks pretty good, menu works fine, I switched from ACPI to DT and it boots into Grub from USB or microSD. I used a NixOS ISO ( After selecting the serial console in Grub and progressing to the kernel it gets stuck, though: Also tried an ISO from another distro with very similar results. Did not get any output on HDMI, but I didn't try very hard. Any idea on how to move on? |
|
I built the NixOS image from @ryan4yin: https://github.com/ryan4yin/nixos-rk3588/tree/main, the rock5a variant, needed a small patch: diff --git a/flake.nix b/flake.nix
index 5102bbc..ecffe1a 100644
--- a/flake.nix
+++ b/flake.nix
@@ -31,7 +31,7 @@
...
}: let
# Local system's architecture, the host you are running this flake on.
- localSystem = "x86_64-linux";
+ localSystem = "aarch64-linux";
pkgsLocal = import nixpkgs {system = localSystem;};
# The native system of the target SBC.
aarch64System = "aarch64-linux";
diff --git a/pkgs/mali-firmware/default.nix b/pkgs/mali-firmware/default.nix
index f2bb1c0..bc48e64 100644
--- a/pkgs/mali-firmware/default.nix
+++ b/pkgs/mali-firmware/default.nix
@@ -7,7 +7,7 @@ stdenv.mkDerivation {
version = "g21p0-01eac0";
src = fetchurl {
- url = "https://github.com/JeffyCN/mirrors/raw/e08ced3e0235b25a7ba2a3aeefd0e2fcbd434b68/firmware/g610/mali_csffw.bin";
+ url = "https://github.com/JeffyCN/mirrors/raw/libmali/firmware/g610/mali_csffw.bin";
hash = "sha256-jnyCGlXKHDRcx59hJDYW3SX8NbgfCQlG8wKIbWdxLfU=";
};
Copied to SD; throws errors due to hardware mismatch but boots fully with network working (on serial console, did not try HDMI). |
Could you elaborate on the steps you took to get this working? I am unable to reproduce. The rawEfiImage fails to build, and the sdImage seems to have a broken partition table (I cannot mount the second partition, and the first one is empty) |
|
My local system isn't an aarch64-linux, so I only changed the mali firmware url. (But it doesn't look like it matters; Then I run (I've been struggling with updating to a more recent kernel, but that's another story...) |
|
@jneem What kernel versions worked for you? |
I tried booting the second two options (armbian/linux-rockchip and mainline 6.10) using armbian (instead of nixos) to build an sdcard image. The bootable armbian sdcards skip edk2 and they boot with working ethernet. |
|
Btw, I have an updated branch here. Compared to @navarro967 's branch, it's based on a more recent master and it comes with some updates to the pin definitions (but I didn't actually observe any differences caused by these changes) |
|
Thanks for your work @jneem! Did you use mainline with just the armbian config or did you also apply their kernel patches? I have tried to build the 6.10.{5,9} Kernels with the armbian patches and a modified version of their config (with compressed firmware loading enabled), but haven't been able to successfully build it yet. I am pretty sure it is an issue with my build process though. I have also tried the armbian 6.1.75 and the friendlyarm 6.1.57 kernels, both boot but no Ethernet. The chip doesn't show in lspci, i haven't done more digging yet. Curiously the friendlyarm kernel with almost the same config (i had to enable efi and dmiid) and the same dts has working ethernet when used in the vendor debian 12 image. I have made a flake to test nixos on the cm3588 a while ago, but just now gotten around to cleaning it up. Here is a link for anyone that's interested. |
|
I've been trying without the armbian patches so far, but maybe I should give them a try. Btw, 6.11 now has mainline support for the cm3588! I'm building that now. Working ethernet is a priority for me, but I was hoping to get a new kernel booting before digging into the ethernet issue... |
If it's the same issue I had, it's because you need to set |
|
I got 6.11 working (with ethernet, but no HDMI for some reason) on my cm3588. Here is the minimal config needed: boot.kernelPackages = pkgs.linuxPackagesFor pkgs.linux_latest;
hardware = {
enableRedistributableFirmware = true;
deviceTree = {
enable = true;
name = "rockchip/rk3588-friendlyelec-cm3588-nas.dtb";
};
}; |
The Crypto instructions on RK3588 are disabled by default. Whether they are enabled in your environment depends on what build of BL31 is being used. See this patch for reference: https://review.trustedfirmware.org/c/TF-A/trusted-firmware-a/+/29363 |
Amazing! All that fiddling with armbian forks and custom configs, and it turns out that latest mainline just works! (For anyone following along, it's important here to be using |
|
I am trying to flash the firmware from this branch (https://github.com/jneem/edk2-rk3588/tree/cm3588), however, I am unable to enter the boot options when pressing escape at the boot splash screen. I've tried 2 different keyboards. It boots into openmediavault by default no matter what I do. Any suggestions? |
|
It looks like only one USB port was being powered up (the upper USB 3.0 port). The latest commit should enable them all. |
|
For the NixOS users here, how did you cross compile jneem's EDK2 branch? |
|
@jneem Thanks a lot for your work! 🤗
Until someone actually answers your question .. I did not want to trial and error the required tools and went the Debian route (on Enable Create In Debian: Find Have not tried the UEFI image yet. I also have a |
When I try to log into the root account and then execute However, after I log in, I execute So something is still off with the network config in particular.
I tried adding the What I'll try next is adding the subnet mask to both the EDIT: I've finally managed to ssh into the device using my non-root account, so thank you for the help! |
Glad you made it work! I would not bother with |
|
Turns out I already had those blocks in my config. If I ever meet you in meatspace, I definitely owe you a beer :) |
|
There's 1 final issue I need to tackle: The nixos config provides LAN access to I can ssh into it, but no internet access, making updating the system a bit difficult. So I create a default route using The only issue is, at the moment it's necessary that I execute it on every reboot, which is undesirable. networking = {
dhcpcd.enable = false;
firewall.enable = false;
hostName = "cumulus";
useDHCP = false;
useNetworkd = true;
interfaces.enP4p65s0.ipv4.routes = [{ # I was hoping this would do the trick
address = "192.168.1.145/24";
prefixLength = 24;
via = "192.168.1.1/24";
}];
};But that just borks the LAN connection, necessitating a reinstall from Armbian. Any idea how I can "nixify" the EDIT: I also tried this, but without success: systemd.services.internetAccess = {
wantedBy = [ "multi-user.target" ];
after = [ "network.target" ];
description = "Add internet access.";
path = [pkgs.bash pkgs.iproute];
script = ''
ip route add default via 192.168.1.1 dev enP4p65s0
'';
serviceConfig = {
Type= "oneshot";
User = "root";
Restart = "no";
};
}; |
|
@jjpe you set |
Everything I mentioned in my last post is with this block in the configuration: systemd = {
network = {
enable = true;
networks = {
enP4p65s0 = {
address = [ "192.168.1.145/24" ];
matchConfig = { Name = "enP4p65s0"; Type = "ether"; };
gateway = [ "192.168.1.1/24" ];
};
};
};
};And this does allow me to ssh in from another host on my LAN. But it doesn't provide internet access for the CM3588, necessitating the manual execution after every boot of That's a bit... suboptimal for a server, especially so since none of my other hosts need any manual network configuration at all. |
The gateway must not have the |
|
Removing the gateway subnet mask made the difference. Thanks, now the real configuration can begin :) |
|
Somewhat related to this thread: does anyone happen to know if the same Looking at the website, it looks exactly like the CM3588 except with more RAM and possibly higher capacity emmc. But I also know that looks can be deceiving when it comes to hardware, so I'm not making any assumptions. |
Yes, mine is a |
|
@sanderhollaar Thank you for this info, that helps a lot! Aside from my |
|
First of all, this is really great work. I'll get a CM3588 next week and am really looking forward to having an alternative to the Debian BSP. However, from the Collabora-Link, it looks like running this board in a mainline 6.11 kernel will not support any of the video de-/encoding features of the board, right? I was hoping I could run Jellyfin there, because they support
Is it possible to boot the verndor's BSP Debian kernel as a workaround (if that's needed)? |
There won't be any mature replacement for RKMPP & RKRGA in the mainline anytime soon, so Jellyfin rewrote RKMPP support in FFmpeg from scratch. If you need normal transcoding functionality, please stay away from the mainline. https://jellyfin.org/docs/general/administration/hardware-acceleration/rockchip |
Do you happen to know the reason for this? |
Because third-party developers do not have enough resources and time to develop extremely complex VPU drivers in the mainline full-time, and because V4L2 cannot meet Rockchip's actual product needs, they will not spend any time in upstream V4L2. Read through this ticket. jellyfin/jellyfin-ffmpeg#34 |
|
Not sure if this question belongs here, but has anyone gotten higher speeds than 100Mb/s on the NIC? I'm running nixos on the 6.11 kernel, and for whatever reason the NIC decides to run in 100Mb/s mode. The only thing I can think of is that it's some limitation of the DTS file in the 6.11 kernel, kind of like how HDMI isn't working either. If anyone has any info on this, please let me know! :) EDIT: Today I Learned that I had a CAT5 cable in my home network setup...... |
|
Hey, Is anyone here able to confirm that all M.2 and USB3 ports work correctly in UEFI & mainline Linux (>=6.10), with latest firmware build from: https://github.com/edk2-porting/edk2-rk3588/actions/workflows/build.yml ? |
|
Hey @mariobalanica I have had some issues with one nvme not showing, but reseating the nvme and the board on the carrier solved them for me. |
|
What about inside UEFI? I don't expect all M.2 slots to work with the v0.12 release because it's using a wrong bifurcation config, but they should with this build: https://github.com/edk2-porting/edk2-rk3588/actions/runs/12530647406 Also, one of the USB 3 ports was limited to 2.0 mode. |
I have the same usecase. Any pointers on how I can setup the vendor bsp kernel with nixos? |
There are several Do you have a pointer to the verndor BSP Debian kernel ? |
Looking at the documentation here: https://wiki.friendlyelec.com/wiki/index.php/CM3588#Build_kernel_linux-v6.1.y, I suspect this is the repo: https://github.com/friendlyarm/kernel-rockchip. Am I on the right track here? |
Something like this. Use
Generating an iso I have no experience with. If you have NixOS installed it might be easier to replace the kernel source with (something like) the above, and keep both generations so you have a known working configuration to go back to. |
|
I'm trying to first setup nixos with the mainline kernel. I've changed the configuration to use device tree mode with mainline support.
The way I am currently generating the ISO is by using the following nix config: I then use the following command to generate the iso: Any idea what I could be doing wrong? |
My guess would be missing kernel modules in the initrd. But it can be anything. Are you able to capture the boot output and post here? You can also install NixOS from the Debian image as described in: #142 (comment) |
I've just reproduced and tested with HDMI, it works fine as well. STR: 1.clone a repo git clone https://github.com/ryan4yin/nixos-rk3588.git
src = fetchurl {
- url = "https://github.com/JeffyCN/mirrors/raw/e08ced3e0235b25a7ba2a3aeefd0e2fcbd434b68/firmware/g610/mali_csffw.bin";
+ url = "https://github.com/JeffyCN/mirrors/raw/libmali/firmware/g610/mali_csffw.bin";
hash = "sha256-jnyCGlXKHDRcx59hJDYW3SX8NbgfCQlG8wKIbWdxLfU=";
};
sh <(curl -L https://nixos.org/nix/install) --daemonreopen shell after installation
recommend to read https://nixos-and-flakes.thiscute.world/development/cross-platform-compilation, before cross-compiling, as I did... nix build .#sdImage-rock5a-cross --extra-experimental-features nix-command --extra-experimental-features flakes
just be carefull with zstd -d result/sd-image/rock5a-sd-image-*.img.zst -o rock5a.img
# increase img's file size
dd if=/dev/zero bs=1M count=16 >> rock5a.img
# check your loops devices before mounting image
ls /dev/loop*
# mount image to loopN
sudo losetup --find --partscan rock5a.img
# check your loops devices after mounting image
ls /dev/loop*
# find out new one, which has 2 partitions, replace loop0 within next commands with loopN you've found
nix shell nixpkgs#parted
## rock 5a's u-boot require to use gpt partition table, and the root partition must be the first partition!
## so we need to remove all the partitions on the sd card first
## and then recreate the root partition with the same start sector as the original partition 2
START=$(sudo fdisk -l /dev/loop0 | grep /dev/loop0p2 | awk '{print $2}')
sudo parted /dev/loop0 rm 1
sudo parted /dev/loop0 rm 2
sudo parted /dev/loop0 mkpart primary ext4 ${START}s 100%
# check rootfs's status, it's broken.
sudo fsck /dev/loop0p1
# umount the image file
sudo losetup -d /dev/loop0
# flash the image to the sd card
cat rock5a.img | sudo dd status=progress bs=8M of=/dev/sdXAlso tested that same image with nanopi-m6, with the only difference: it's u-boot partition is 8M long, instead of 16M for nanopc-CM3588-nas, and I've been writing u-boot part, copied from DietPi-NanoPi-M6 PS: for ppl, who doesn't wanna mess around with nix-os and just want a working rockchip-based SBC, I recommend this: https://dietpi.com. Their images are pretty lightweight and fast and as live-cd images for SD cards they just work perfectly without any additional complicated things required... |
@NomisIV Would you mind posting the config? I have NixOS working on eMMC, and I'm trying to get NixOS working on ZFS, and while my RAIDZ1 array itself works (also 4 SSDs, on NixOS-on-eMMC I can manually import my zpool and then use the datasets as normal, including the ones I want to use to re/install NixOS on), I haven't been able to come up with a working NixOS system config that allows booting from the ZFS array. My hypothesis is that I have some small flaw in my config somewhere, and a point of reference there would be quite helpful. |
|
@jjpe I think this is the only really relevant part: https://codeberg.org/NomisIV/dotfiles/src/branch/master/machines/flash.nix#L3-L9. Well, that and the filesystem block a bit further down. Something to note is that I have the /boot partition on the eMMC (in another partition after the 8M edk-rk3588 partition) and not the ZFS array. I hope this helps :) |
What would be required to add support for the FriendlyElec CM3588 NAS kit board? I have very little experience in this area, but I am motivated to get this implemented. I do own this board myself, so I would be able to perform testing on it.
What I've figured out myself is that the device tree (dts) file needs to be included here. The proper device tree for this board is probably this one.
Are there any more files that need to be included for proper support?
Looking forward for input!
The text was updated successfully, but these errors were encountered: