RPM Lint and Mock

This post is very old; Ive had it sitting as a draft for ages.

rpmlint is a tool for checking common errors in rpm packages. It can be used to test individual packages and spec files before uploading or to check an entire distribution. By default all applicable checks are processed but specific checks can be performed by using command line parameters. 

I love man pages; they always have the best descriptions. rpmlint is one of the most sensitive tools I’ve ever used on Linux. One little thing and it warns you even if its commented out. In the users perspective that’s a good thing since it sort of forces developers to create better installation scripts; however developing an RPM to get warnings for some pretty useless things gets not necessarily frustrating but really annoying.

Ive built 3 RPM packages recently:

1. Installation of a yum repository (special thanks to rpmfusion)
2. 64-bit Firefox nightly
3. 32-bit Firefox nightly

The amount of time i spent on making these RPMs was actually fairly short considering the emo tools that have to be used to validate them. Both Firefox nightly RPMs are the same (save for the source tar ball).

Mock is  a simple program that will build source RPMs inside a chroot.
It doesn’t do anything terribly fancy other than populate a chroot with
the  contents  specified  by a configuration file, then build any input
SRPM(s) in that chroot.

I haven’t worked with Mock too much however from what i can tell so far; it is a fairly strong tool however just as annoying as rpmlint if not more. The tool just compiles a program with the default packages just like the description above says however what it doesn’t tell you is that it also checks for really small things such as the use of wild cards.

Now lets see how mock works; you must build a package and its source RPM (rpmbuild -ba ~/rpmbuild/SPECS/file.spec) and then run mock for a preferred operating system (mock -r fedora-13-x86_64 ~/rpmbuild/SRPS/file.src.rpm) then see the output for any errors; Heres the kicker; for you to create a source RPM the package must build correctly; therefore you assume that everything is fine minus any non-default packages.

File not found by glob: /builddir/build/BUILDROOT/minefield-4.0b8pre-1.fc14.x86_64/usr/lib64/minefield-4.0b8pre/*

I got this error every time i used mock; I tried to fix it about a dozen times and i just couldn’t figure out where this issue was in the spec file. After a couple frustrating hours i figured out that it was the wild card being used in the %files section of the spec file.

After fixing that ridiculous issues; mock successfully completes; but just because it completes doesn’t mean everything works; you now have to comb through a log file and attempt to spot any errors.

Release 0.3: Update

The semester is wrapping up and we’ve done a lot of work over the last couple of weeks to set everything right. Ive attempted to get as much work done as i can however the amount of work was sometimes overwhelming. As the project stands now, the system is working however not complete. Ill post some of the changes that we’ve made since release 0.2 as well as future updates that need to be made.

Repository RPM:

• Bumped version to 0.4
• Cleaned up .repo file
• Removed invalid gpg key
• Fixed a typo

Repository:

• Includes drpm and srpm directories which hold Delta RPMs and Source RPMs respectively.

Minefield RPM:

• Passes rpmlint; I did say that it passed in my release 0.2 however that was only for the spec file. The rpms output 2 warnings: no documentation; no manual
• Both 32 bit and 64 bit source RPMs are now in the repository

Minefield Spec file:

• Removed a buildrequires (tree)
• Removed use of the “install” command because of permission issues and replaced with “cp” which can copy files/directories with their file permissions

Automation:

• Script now takes URL as the argument; version number and architecture are taken from the URL.
• Script validates configuration
• Removes partial wget downloads
• Performs local jobs first before going out to download minefield

The main goal for this release was getting automation to a point where it can be implemented into a system and not have to worry about any issues caused by future changes. Achieving this is relatively difficult however Ive made it so that the only thing that needs to change is the URL to access the latest minefield. Ive talked about how to set up the server side of our program here.

don’t worry about integrating anything into Mozilla’s systems this
quarter as I pretty much have to switch gears to finish another couple
of goals.

Although our project page said that our goal for this release would be to have our system implemented into Mozilla’s infrastructure; it looks like it wont. I have blogged about everything to get the system working on their side so i have done my part. It is now up to them to implement it. I will be happy to assist them in doing so and to make any improvements to what we have developed.

Future updates to be done:

• Complete automation scripts
  • validation is incomplete
  • only tested for nightlies
• Implement RPM signing
• Fix any outstanding bugs. Ive talked about bugs here

Recent Changes

The automation scripts have been disabled for a couple days because i was planning on making some major changes to them. I made some changes to it tonight but I’m not done with it yet. Expect updates in the very near future. Some of the changes I’ve made so far:

• Version number changes:
  Old: minefield-4.0b8pre-20101211.fc13.rpm
  New: minefield-20101211.1-4.0b8pre.i686.rpm

  The versioning system was changed to make the rpm version number valid. The version number is not allowed to have any letters so i just switched the version and the release numbers. At the end of the version i added a .1 to add support for releasing multiple nightlies in one day. The .1 will increment if there are more than 1 release on the same day. The fc13 part was removed as this release isn’t tied to any release of Fedora; since the program we packaged is pre-compiled it should theoretically work on most if not all versions of Fedora.

• Delta RPMs:
  drpm files will now be generated; this will reduce the download size for users updating from an older nightly.
• Source RPMS:
  src.rpm files are placed in a sub directory under the correct architecture.
• Spec File:
  The use of the command “install” was removed; it has been replaced by the “cp” and “mkdir” commands. The reason I did this was because cp is capable of keeping file permissions the same without having to specify it.

  A buildRequires was removed as a result of switching out the tree command with cp. The spec file no longer installs one file at a time; it simply copies the entire directory to the correct location.

  Although it was fast enough; the above changes have significantly improved the install speed of the application.

All of the older nightlies with the old versioning has been removed. To take advantage of the new system; remove your current minefield and install the latest one with YUM.

Bugs

Armen, our contact at the Mozilla release engineering team has posted about our progress on the project on his blog. I’m honored that I could be part of a team which developed a great application and that my work can somehow be used in the future and further developed by others. Unfortunately there are issues with what we have developed and I would like to address them here. I have pulled the list of issues and problems that Armen posted on his blog as well as other issues that I have found with my own testing.

• Current and previous Minefield nightly show up under “Add/Remove Software”

Ive been looking into this after reading Armen’s blog and I’m not quite sure what is causing it. I am fairly confident to say that it isn’t on our side. It may be a bug in the “Add/Remove Software” application or a way Fedora implemented their filters. I updated another application on my system that is on the official repositories and it too shows up twice.

• Setting Minefield as the default browser seems to mess up the “Preferred Web Browser” icon.

• Using the “Preferred Web Browser” launcher opens the users home directory.

Using the “Default Browser” dialog simply doesn’t work; I highly recommend not using that dialog. If you wish to set Minefield as the default browser navigate to System -> Preferences -> Preferred Applications. To fix the above 2 issues, it should match the following:

When using the “Default Browser” dialog, the command is set to: /usr/lib/minefield-4.0b8pre/firefox “%s”; The quotation marks cause the browser to open the home directory; removing the quotations will fix that issue alone. The other issue which is the “Preferred Web Browser”‘s icon; the “Command” field must not be an absolute path.

After changing this; The “Preferred Web Browser” will automatically change to the same exact command as the one that opens the menu item.

• Should we be listed under “debug and development software sources” on the Software Sources manager?

Although it would make sense to put it there; the repository is installed by the user and should be listed as a regular repository. Unless this is really bothersome, I will look into changing it.

• Minefield gets unpacked under /usr/lib/minefield-4.0b8pre. Should this be instead /usr/lib/minefield?

The version number is in the unpacked directory’s name to keep some level of consistency with other packages on the system as well as Firefox itself. The Firefox executable points to /usr/local/lib/firefox-4.0b8pre. Firefox itself is compiled with that directory. The reason files are not placed in /usr/local/lib is so that we follow at least some of the Fedora guidelines for packaging software.

• Minefield when launched from Applications->Internet->Minefield starts with the ProfileManager and allows running the nightly build with the other instances of Firefox (“minefield –no-remote –ProfileManager); should this not be a feature?

Personally I don’t like this feature as I can imagine it would get really annoying. However it was suggested by Chris to be implemented to help out new users with using the nightly. I leave this option to Armen whether to keep or not.

To remove this change:

• edit /usr/share/applications/minefield.desktop
• remove “–no-remote –ProfileManager” from the end of the line which starts with “Exec”

• Since /usr/lib/minefield-4.0b8 is owned by “root” Firefox’s update system doesn’t work. If we change ownership then updates work again and I wonder what bug would I hit if I can receive updates through yum and through AUS. Only one way to find out!

Updating the nightly from yum will remove the directory and recreate the directory. This will ensure that even if the version bumps up, the old directory will disappear and the new directory will appear in its place. From the best of my knowledge using AUS should not cause any issues unless the version number changes. I would highly recommend using one or the other and not both.

• I also believe that there is not a way to offer partial updates through yum as we do through our normal update system

Delta RPMs will allow for partial updates. It is roughly the equivalent to the way patch files are with the diff command. I will implement this into the repository as well as the automation scripts as soon as i can.

• Minefield is being listed in the wrong group. It is listed as “Other” however it should be listed under “Internet”

No idea whats causing this; The spec file specifies the group exactly the same way as other packages but it still shows up under “Other”.

Setting up the Server

Although we haven’t implemented automatic signing or any signing at this point, I wanted to give something for you to work with. Ive made some last minute changes and some comments to the scripts just to ease the learning curve of using these scripts.

Ive uploaded a copy of the automation scripts to “http://tarin.freehostia.com/files/scripts.tar.gz”. The steps to actually getting this to work is as follows:

1. Create a new user. (Not required but highly recommended; the scripts remove files in the home directory / repository)
2. Download the scripts from http://tarin.freehostia.com/files/scripts.tar.gz
3. Extract the tar ball into the home directory of the new user (or current user if u decided to use it)
4. Verify that the directory structure is as follows:
   • ~ (home directory)
     • UpdateRepo.bash
     • includes
       • BuildDesktopFile.bash
       • BuildSpecFile.bash
     • …
5. edit UpdateRepo.bash; The top of the file holds all the configuration (there’s only 1 thing to configure)
   Change the path to the parent directory of your repository. (do not end with a trailing slash)
6. Run the script with the following commands:
   ~/UpdateRepo.bash http://ftp.mozilla.org/pub/mozilla.org/firefox/nightly/latest-trunk/firefox-4.0b8pre.en-US.linux-i686.tar.bz2
   ~/UpdateRepo.bash http://ftp.mozilla.org/pub/mozilla.org/firefox/nightly/latest-trunk/firefox-4.0b8pre.en-US.linux-x86_64.tar.bz2

Troubleshooting:

Q: I run the script but i get “Bad Configuration: * unwritable”. Help!
A: Verify that the new user has access to write to your repository.

Q: I run the script but i get “Missing Dependency: rpmdevtools”. Help!
A: To fix this, login as root and type in: “yum install rpmdevtools -y”. This should install required packages.

Q: I have a different problem!
A: The scripts are currently experimental, any issues or errors should be posted here.

EDIT: Fixed links