Fixing Maverick Crunch FPU code generation
in GCC 4.1.2 and 4.2.0

Martin Guy <martinwguy@yahoo.it>
25 August 2007
Last updated: 18 December 2008

News

Preamble

The Maverick Crunch code generation in GCC has never worked. Among other things, it cannot reliably compare two floating point numbers. For details, see the file compare-bug.

To use this you need to be using an ARM EABI kernel and file system such as the new Debian "armel" port or the Angstrom distribution and you need to build your own fixed version of GCC.

System requirements

gcc-4.1.2 will build in 64MB (just!). If you have some swap space, so much the better.

To build gcc-4.2.0 with full optimisation you need more than 64MB RAM; if you only have 64MB you can work around this with some manual intervention, described below.

With 64MB it gets into trouble when it arrives at gcc/insn-recog.c: if you have no swap space, gcc will die horribly and/or the kernel will start killing processes. If you do have some swap space it will take forever, thrashing pages in and out of swap and getting nowhere. You can see this happening by launching "vmstat 5" on another console and watching the "si so" columns go into the hundreds and the "us" column go down to zero.

You can get past the file that requires more than 64MB of virtual memory by waiting until the compilation gets to insn-recog.c (or until it fails if you have no swap), then use "make CFLAGS=-g" to compile the problem file without optimisation, and when it has succeeded on that file, interrupt it and continue with plain "make".

Alternatives are to build it on an ARM simulator such as QEMU where up to 247MB of simulated RAM can be configured, or to use a cross-compiler on a big computer; these options are not covered here.

Fixing and building gcc

• They disable all non-branch conditional instructions on all ARM processors, which may increase code size
• They may break code generation for other ARM processors
• There is known to be a problem if you compile glibc libmath: tanh(double) fails.
• Though they work on revision E1 silicon, revision E2 gives problems.

Here is the original tarball from futaris.org, of which there is also an unpacked browsable copy on this machine and here are single files containing all the necessary patches in one lump for gcc-4.1.2 and for gcc-4.2.0.

I built a native compiler on the ARM box itself running the "armel" Debian ARM EABI port.

Here's how to make a Crunch-capable version of gcc-4.1.2. To make gcc-4.2.0, just s/4.1.2/4.2.0/.

• fetch the gcc-4.1.2 tarball from a GCC mirror site and unpack it.

      wget ftp://sourceware.org/pub/gcc/releases/gcc-4.1.2/gcc-4.1.2.tar.bz2
      tar xjf gcc-4.1.2.tar.bz2
  

• fetch the futaris crunch tarball and unpack it (it unpacks into three subdirectories: binutils, gcc and glibc)

      wget http://files.futaris.org/gcc/crunch.tar.bz2
      tar xjf crunch.tar.bz2
  

• Apply patches from the futaris directory gcc/gcc-4.1.2 that are listed in the SRC_URI section of the gcc/gcc_4.1.2.bb BitBake file. (There are many versions of some of the patches in that directory; you only want the versions that are listed in the BitBake file).
  For convenience I edited the list of filenames into a file called gcc/gcc-4.1.2/series. Once I'd done that, I could apply all the patches in one shot:

      cd gcc-4.1.2
      for a in `cat ../gcc/gcc-4.1.2/series`
      do
  	patch -p1 < ../gcc/gcc-4.1.2/$a
      done
      cd ..
  

  (One patch in the gcc-4.2.0 SRC_URI list, fix-ICE-in-arm_unwind_emit_set.diff, has already been applied in the mainline gcc tarball. "Patch" reports it as an already-applied or reversed patch: just say No :)
• Configure gcc. I do this for just C language, configured the same way as the existing Debian compiler supplied with armel, adding
  • --disable-bootstrap so that it uses the system compiler to build the new compiler (instead of first compiling a minimal version of itself using the system compiler and using *that* to build the final compiler).
  • --enable-languages=c because I only need C for my tests. If you omit this you get compilers for all possible languages, or you can give --enable-languages=c,c++ or whatever.
  • --program-suffix=-4.1.2-futaris so that the final compiler will be installed as /usr/local/bin/gcc-4.1.2-futaris I need to do this because I am comparing several different versions of Maverick Crunch patches for different version of gcc and need to be able to call them all by different names.
  • arm-futaris-linux-gnueabi is the main configuration tuple. The "futaris" here can be anything; it is the "vendor string" that is used to separate the installed files of the same version of the compiler obtained from different suppliers. I use it to distinguish the futaris-modified version of gcc from the Cirrus-modified version of the same compiler.

      mkdir build-4.1.2
      cd build-4.1.2
      ../gcc-4.1.2/configure --disable-nls --enable-shared --with-system-zlib \
  	--without-included-gettext --enable-threads=posix \
  	--enable-clocale=gnu --enable-mpfr --disable-libssp \
  	--disable-bootstrap --enable-languages=c \
         	--program-suffix=-4.1.2-futaris arm-futaris-linux-gnueabi
  

• Build the compiler

      make
  

• Install it (it will go into /usr/local/bin, /usr/local/lib and so on)

      make install
  

• I also run the IEEE-754 floating point tests on the compilers I made, for which the runes are:

      make check-gcc RUNTESTFLAGS="ieee.exp --target_board=unix/-mcpu=ep9312/-mfpu=maverick/-mfloat-abi=softfp"
  

Using the new compiler

    gcc-4.1.2-futaris -mcpu=ep9312 -mfpu=maverick -mfloat-abi=softfp -O2 foo.c

    ./configure CC=gcc-4.1.2-futaris CFLAGS="-O2 -mcpu=ep9312 -mfpu=maverick -mfloat-abi=softfp"
    make
    make install

• -mcpu=ep9312 selects the combination of arm920t instructions plus Maverick Crunch instructions.
• -mfpu=maverick selects Maverick Crunch FP instructions, as opposed to VFP or FPA
• -mfloat-abi=softfp is half way between -mfloat-abi=soft (a synonym for -msoft-float) and -mfloat-abi=hard (ie -mhard-float). With softfp, real floating point instructions are used to perform FP calculations inside function bodies, but floating point function arguments and return values are passed in integer registers in the same way as it is done with soft-float. This allows us to use real floating point code but still link it with libraries that were compiled for soft-float, as is the case for all Debian armel libraries. True hard-float is slightly more efficient since it passed floating point values in FPU registers, but can only be used if the entire system, all executables and all libraries have been compiled to use hard-float; for Debian that means creating a whole new port.

Trying to set Maverick Crunch as the default compiler target

One of the gcc patches, unbreak-armv4t.patch already changes the default instruction set from armv5 to armv4. Without this, the gcc build dies half way saying "Illegal instruction".

Other flags that could be used in the gcc configuration line are --with-cpu=ep9312 --with-fpu=maverick and presumably --with-float-abi=softfp but I never found a combination that would work with all the GCC components;

• configure ... ep9312-futaris-linux-gnueabi immeditely says "machine `ep9312-futaris' not recognized"
• If I remember correctly, using configure --with-cpu=ep9312 at runtime the assembler complain that ep9312 is not a known processor
• and using configure --with-cpu=arm920t --with-fpu=maverick the assembler complains that the selected processor does not support Maverick Crunch instructions (the same as it does if you use "gcc -mcpu=arm920t -mfpu=maverick").

Alternative patch sets

It unpacks to three shell scripts: 1-download.sh, 2-unpack.sh and 3-build.sh and contains patches for the abovementioned tools. I've applied their gcc patches to gcc-4.1.2, built and tested it as above, and it fails 15 of the tests in gcc's IEEE test suite (while the futaris patches pass 100% of the tests). With the necessary -ffix-crunch-d1 flag, it fails 14 of the tests.

However, it does seem to work with a couple of small testpieces that spring bugs in unpatched gcc, and LAME seems to work ok with it and at the same speed as the futaris-patched versions, give or take 1%; the output file is playable but is different from the output produced by a correct FP implementation.

Speed tests

Except where -O flags are specified, LAME was compiled -O2

Or, to summarise

• soft-float (Debian armel) is 11.5 times faster than emulated FPA (old Debian arm)
• Maverick Crunch softfp is 2.4 times as fast again
• soft-float with fastmath is slower than plain soft-float
• size or speed optimisation or using different compiler versions makes very little difference to the speed of a floating point-intensive task once you are using the FPU.

Update 22 Oct 2007

The only floating point option to pass all the "paranoia" tests on Cirrus hardware is soft float.

Conclusion: there are still no patches to gcc that generate reliable Maverick Crunch code.

Interestingly, the standard gcc for x86 can compile paranoia.c but fails many of the tests. Without optimisation, gcc-3.4 and gcc-4.1.2 have 1 defect and 1 flaw; with -O2 gcc-4.1.2 has 3 failures, 4 serious defects, 3 defects and 2 flaws.

Update 11 Dec 2008

In April, Hasjim sent me a twisty little maze of patch ideas for gcc 4.2.1, 4.2.2, 4.2.3 and 4.3.0, all different. I've tried and compared them, analysed and fiddled with them and have cooked up a set for 4.3.2.

The best maximum-speed options are currently:

  -mcpu=ep9312 -mfpu=maverick -mfloat-abi=softfp -mcirrus-fix-invalid-insns -O2 -mfast-math

The strategies are:

• Don't use conditionally-executed floating point instructions, thus avoiding a shoal of timing-dependent hardware bugs that trash results and/or memory.
• Drop the cfcmp64 integer 64-bit comparison instruction, because it cannot do the simultaneous signed and unsigned comparison that GCC expects. 64-bit integer comparisons are done in pairs of ARM registers instead.
• Don't use the Maverick registers the 32-bit integer modes, since the ARM registers are so much richer in that department.
• I have dropped the "cirrus-bugfixes" patch that Hasjim includes, because it did not appear to do anything. This reinstates mainline's -mcirrus-fix-invalid-insns flag, which does appear to emit the right number of nops in the right places after branch instructions so as to avoid the "fp instruction not executed because in the two instruction slots following a taken branch" clause of certain hardware bugs.
  That said, my tests on E1 revision silicon fail to trigger bug 4 at all, and my test for bug 2 only shows it being be triggered by an appropriate FP instruction in the first location following a taken branch, not also in the second as the errata say. However, that may be due to a fortunate interlock wait or to the target instruction not being cached already.
• I reduce the amount of nop emission in the function epilogue and elsewhere, and the correctness tests I have tried (FFTW, LAME) still seem to pass.
• IEEE-perfect math is impossible in all existing EP93xx silicon (and no fixed silicon is ever going to be released) because one of the hardware bugs make the floating point add, sub, abs, neg and cpy instructions take tiny denormalised numbers on their inputs as zero thereby dropping values in the range from ±2^-149 to 2^-126 for floats and from ±2-¹⁰⁷⁴ to 2-¹⁰²² for doubles.
  I work round buggy cpy by using one of the literal 64-bit copy instrutions; Hasjim disables some of the other instructions so that their accurate equivalents are performed in pairs of ARM registers.

  Since we can't ever attain perfect IEEE math without disabling the FP add and sub instructions too, I prefer to declare a precision of ±2^-1022 for doubles and enable all the denorm-unsafe instructions for maximum speed.

• Implement the CCMAV mode for correct condition code interpretation.
• Instead of disabling the denorm-mashing instructions in the initial patches then re-enabling them in a final one, just don't disable them.
• Implement a -mieee switch, to disable all the imprecise instructions and give full precision.

TODO

• Port the same stragegy as patches for gcc 4.1 and 4.2.

We discuss these things on the linux-cirrus mailing list, carbon copied to the GCC groups and others when appropriate.