Topic: Hard Fault during certificate verification on LPC1768 using GCC-ARM

I have an application that I prototyped w/ the MBED on-line compiler and it runs nicely. I exported the project and now I am compiling it w/ GCC (gcc-arm-none-eabi-4_8-2013q4) and I am getting hard faults during certificate verification. The following is the output of the program with DEBUG_WOLFSSL defined. The last part is the output of my hard fault handler:

wolfSSL Entering SSL_connect()
growing output buffer

Shrinking output buffer

connect state: CLIENT_HELLO_SENT
growing input buffer

growing input buffer

received record layer msg
wolfSSL Entering DoHandShakeMsg()
wolfSSL Entering DoHandShakeMsgType
processing server hello
wolfSSL Leaving DoHandShakeMsgType(), return 0
wolfSSL Leaving DoHandShakeMsg(), return 0
growing input buffer

received record layer msg
wolfSSL Entering DoHandShakeMsg()
wolfSSL Entering DoHandShakeMsgType
processing certificate
Loading peer's cert chain
    Put another cert into chain
    Put another cert into chain
    Put another cert into chain
wolfSSL Entering GetExplicitVersion
wolfSSL Entering GetMyVersion
wolfSSL Entering GetAlgoId
Getting Cert Name
Getting Cert Name
wolfSSL Entering GetAlgoId
wolfSSL Entering DecodeCertExtensions
wolfSSL Entering DecodeSubjKeyId
wolfSSL Entering DecodeAuthKeyId
wolfSSL Entering DecodeBasicCaConstraint
wolfSSL Entering GetAlgoId
About to verify certificate signature
Hard Fault! 40000000
Forced Hard Fault:
     SCB->CFSR = 0x00008200
     SCB->BFAR = 0x0cb85502
r0  = 0x00000003
r1  = 0x00000000
r2  = 0x00000063
r3  = 0x0cb85502
r12 = 0x92492a47
lr  = 0x00000000
pc  = 0x0002881a
psr = 0x81000000

The contents of SCB->CFSR indicate a precise bus error. The program counter value: 0x0002881a is located in the following assembly:

   28816:       e058            b.n     288ca <fast_s_mp_mul_digs+0x126>
      int      tx, ty;
      int      iy;
      mp_digit *tmpx, *tmpy;

      /* get offsets into the two bignums */
      ty = MIN(b->used-1, ix);
   28818:       68bb            ldr     r3, [r7, #8]
   2881a:       681b            ldr     r3, [r3, #0]
   2881c:       1e5a            subs    r2, r3, #1
   2881e:       6bfb            ldr     r3, [r7, #60]   ; 0x3c
   28820:       429a            cmp     r2, r3
   28822:       bfb8            it      lt
   28824:       4613            movlt   r3, r2
   28826:       623b            str     r3, [r7, #32]

The offending code is line 2913 of "integer.c" (fast_s_mp_mul_digs function) in the 2.7.0 release. I'm reading "load r3 into r3" because of the #0 offset? The contents of r3 are not a valid memory address, which I am assuming is causing the bus error.

Any ideas why this might be? Do you even support cortex-m3 compiled with GCC? I could not find any information about that. Here  are the relevant config values from settings.h

#define MBED

#define WOLFSSL_LWIP

#ifdef MBED
    #define SINGLE_THREADED
    #define WOLFSSL_USER_IO
    #define NO_WRITEV
    #define NO_DEV_RANDOM
    #define NO_SHA512
    //#define NO_DH
    #define NO_DSA
    #define NO_HC128
    #define NO_FILESYSTEM
    #define DEBUG_WOLFSSL
    #define SSL_CFG_PSK_EN
    #define NO_DES3 // 3 des is being phased out
    #define NO_MD4 // MD4 is broken and shouldn't be used
    #define SMALL_SESSION_CACHE
    //#define WOLFSSL_CALLBACKS
    //#define USER_TIME
    //#define USE_FAST_MATH // assembler implementation of bigInt routines
    //#define TFM_TIMING_RESISTANT // uses less memory when fast math is enabled
#endif /* MBED */

Share

Re: Hard Fault during certificate verification on LPC1768 using GCC-ARM

Hi mtilchen,

We have had several users who have cross compiled wolfSSL embedded SSL for ARM platforms using GCC.

Do you know what your stack and heap sizes are set at?  Oftentimes we'll see hard faults like this when one of these is not set high enough.  For reference, I have attached the wolfSSL Resource Use document which shows typical runtime memory usage of wolfSSL per session.

One thing which can also make a difference here is what math library you are using.  WolfSSL let's users choose between two different math libraries - the normal big integer (which it looks like you are using) and the fastmath library. 

By default, wolfSSL uses the normal big integer math library.  This is generally the most portable and generally easiest to get going with.  The negatives to the normal big integer library are that it is slower and it uses a lot of dynamic memory.  The fastmath library on the other hand uses less dynamic memory and moves it to the stack instead, also providing speed improvements (using assembly optimizations where possible).

You can enable fastmath by defining USE_FAST_MATH, and since the stack usage can be high, we also recommend defining TFM_TIMING_RESISTANT as well.

Best Regards,
Chris

Re: Hard Fault during certificate verification on LPC1768 using GCC-ARM

Thanks for the advice and for pointing out the different math implementation options. I have tried setting the configuration to use the fastmath but I keep getting errors at various stages of certificate verification all related to ASN data parsing/formatting. It looked like I was running out of stack and I noticed that the stack size for the RTOS threads in the mbed RTOS are set to 512 bytes. I tried increasing this up to 3K. The previously mentioned errors went away but were replaced with explicit memory errors from the library (-125, -132).

I would like to take a look at the "Resource Use" document you mentioned but I do not see it posted here. Nor can I find it anywhere on the website.

Share

Re: Hard Fault during certificate verification on LPC1768 using GCC-ARM

Hi,

It looks like the Resource Use document didn't attach correctly.  I just sent it over to you in an email.

Thanks,
Chris

Re: Hard Fault during certificate verification on LPC1768 using GCC-ARM

I got the document and it was helpful. With the 32k available on the LPC1768 things are pretty tight. Especially when you consider that I am using 2048-bit keys and the server has a 3 cert chain.

I ended up getting things working with the Keil compiler with version 2.7.0. I have not yet dealt with GCC. I actually started with version 2.8.2 but there is an extra 4k (roughly) of ZI static allocation taking place that is not present in 2.7.0 release. The memory map emitted by the linker shows the allocation is taking place in the "ssl.o" object but when I took a quick look at the source I cannot find the code that might be responsible. Any ideas? This allocation is the difference between running out of memory or not in my application.

I now plan on attempting to get the GCC version running as well. I will report my findings here.

Share

Re: Hard Fault during certificate verification on LPC1768 using GCC-ARM

Hi,

We now have version 2.9.0 released, which may warrant an upgrade - so you are working with the most current version.

Can you verify that your settings.h file from both version match up for the sections which are enabled?

Thanks,
Chris

Re: Hard Fault during certificate verification on LPC1768 using GCC-ARM

I went ahead and pulled down the project from Github and checked out the 2.9.1 tag. I have found that the additional static allocation (~4k) over version 2.7.0 is from the session cache and its related structures. I had to explicitly define NO_SESSION_CACHE or SMALL_SESSION_CACHE to get the static allocation back down. I am not sure why I did not have to explicitly define these with the previous version.

I do not use the server-oriented capabilities of the library, creating only client connections. What am I giving up if I don't use any session caching? Renegotiation?

Share

Re: Hard Fault during certificate verification on LPC1768 using GCC-ARM

We did add in a client-side session cache which wouldn't have been in the 2.7.0 release.  This could explain the increase in stack usage.  The client session cache is enabled by default, unless NO_CLIENT_CACHE is defined.

The client-side session cache was added to store server ID's.  This allows the SSL library to store the mapping of wolfSSL sessions to server ID's so that applications no longer need to keep their own mappings.

Best Regards,
Chris

9 (edited by amitsehgal 2014-11-11 16:05:18)

Re: Hard Fault during certificate verification on LPC1768 using GCC-ARM

Did you get it working by any chance ? Can you share your settings.h for mbed on LCP1768?

Hi Chris,

Can you send resource use document to me as well ?

Thanks,
Amit

Share

Re: Hard Fault during certificate verification on LPC1768 using GCC-ARM

Hi Amit,

I just emailed you the Resource Use document.

Best Regards,
Chris