Fixing rpmostree_finalize_kernel hack

[Jonathan Dickinson <oss jonathan dickinson id>]

Hi everyone,

I am interested in fixing the hack that can currently be found on line 426 of rpm-ostree-kernel.cxx <https://github.com/coreos/rpm-ostree/blob/86070c18996fd43c4a2d04012edfe2c9ad15dafb/src/libpriv/rpmostree-kernel.cxx#L426>. I have never worked with this codebase before, but I'm happy to give it a try.

Background

The current behavior prevents the installation of custom kernels authored by "rmnscnce" on COPR <https://copr.fedorainfracloud.org/coprs/rmnscnce/kernel-xanmod/>, because the .vmlinuz.hmac file contains an entry along the lines of `/lib/modules/5.16.12-xm1.4.fc35.x86_64/vmlinuz`. This arises because the .spec calls sha256sum directly on that file. These custom kernels install fine on many other RPM-based distros.

These kernels are extremely popular for scenarios such as waydroid and gaming.

Looking at the stock kernel, the two hmac files are identical (both containing `/boot/...`). This means that `rpmostree_finalize_kernel` treats them both the same way and essentially ends up performing a basename on the path contained the hmac file.

I have informed "rmnscnce" about a workaround. Even if they go ahead and use the workaround, this is a difference in behavior and it should probably be addressed.

Proposal

Given that a basename is the current emergent behavior, I propose formalizing that. Any thoughts on this?

--

 Jonathan Dickinson (he/him)