5

I'm trying to develop a new feature for an embedded application and I'd like to do so using a test-driven approach.

The project is written in pure C and is being developed using IAR Embedded Workbench 6.60.1.5104. I'm targeting an LPC1788, which is a Cortex-M3 device, and all development is being done on a 64-bit Windows 7 machine. Right now I'm more in favour of getting unit tests to run on the PC rather than on the target hardware (RAM is quite limited).

I came across a useful book on the subject called Test Driven Development for Embedded C and that pointed me towards tools like Unity, CppUTest, Ceedling, etc. After looking into this stuff, I think my best choice is to configure Ceedling (which uses Unity) for my project. However, I'm not sure exactly what steps I need to take to configure Ceedling to work with my current IAR toolchain.

I've installed Ceedling and created the "blinky" example project and I'm trying to build and test it using the IAR toolchain. I've added iccarm.exe to my path and edited blinky/project.yml as given below:

---

# Notes:
# This is a fully tested project that demonstrates the use
# of a timer ISR to blink the on board LED of an Arduino UNO
:project:
  :use_exceptions: FALSE
  :use_test_preprocessor: TRUE
  :use_auxiliary_dependencies: TRUE
  :build_root: build
  :release_build: TRUE
  :test_file_prefix: test_

#You'll have to specify these
:environment:
  - :mcu: atmega328p
  - :f_cpu: 16000000UL 
  - :serial_port: COM8  #change this to the serial port you are using!!!
  - :objcopy: avr-objcopy
  # Uncomment these lines if you are using windows and don't have these tools in your path
  # - :path:
    # - C:\mingw\bin
    # - C:\WinAVR-20100110\bin
    # - C:\WinAVR-20100110\utils\bin
    # - #{ENV['PATH']}

:extension:
  :executable: .bin

:release_build:
  :output: blinky

:paths:
  :test:
    - +:test/**
    - -:test/support
  :source:
    - src/**
  :support:
    - test/support

:defines:
  # in order to add common defines:
  #  1) remove the trailing [] from the :common: section
  #  2) add entries to the :common: section (e.g. :test: has TEST defined)
  :commmon: &common_defines []
  :test:
    - *common_defines
    - TEST
  :test_preprocess:
    - *common_defines
    - TEST

:tools:
  :release_compiler:
    :executable: avr-gcc
    :arguments:
      - ${1}
      - -DTARGET
      - -DF_CPU=#{ENV['F_CPU']}
      - -mmcu=#{ENV['MCU']}
      - -Iinclude/
      - -Wall
      - -Os
      - -c
      - -o ${2}
  :release_linker:
    :executable: avr-gcc
    :arguments:
      - -mmcu=#{ENV['MCU']}
      - ${1}
      - -o ${2}.bin

:cmock:
  :mock_prefix: mock_
  :when_no_prototypes: :warn
  :enforce_strict_ordering: TRUE
  :plugins:
    - :ignore
  :treat_as:
    uint8:    HEX8
    uint16:   HEX16
    uint32:   UINT32
    int8:     INT8
    bool:     UINT8

:tools:
  :test_file_preprocessor:
    :executable: iccarm
    :name: 'IAR test file preprocessor'

  :test_includes_preprocessor:
    :executable: iccarm
    :name: 'IAR test includes preprocessor'

  :test_compiler:
    :executable: iccarm
    :name: 'IAR test compiler'

  :test_linker:
    :executable: iccarm
    :name: 'IAR test linker'

  :release_compiler:
    :executable: iccarm
    :name: 'IAR release compiler'

  :release_linker:
    :executable: iccarm
    :name: 'IAR release linker'

:plugins:
  :load_paths:
    - vendor/ceedling/plugins
  :enabled:
    - stdout_pretty_tests_report
    - module_generator
...

The only difference between this and the default project.yml is the content under the second :tools section.

My guess is that I'm heading in the right direction, but I'm not sure whether iccarm.exe is the correct executable to use for all these parts of the toolchain and what arguments I need to pass.

If I can configure Ceedling to build and test the blinky project using an IAR toolchain, I'm hoping I should be able to apply the same configuration for my actual project. If I try running rake now, I get the following output:

$ rake


Test 'test_BlinkTask.c'
-----------------------
rake aborted!
Errno::ENOENT: No such file or directory @ rb_sysopen - build/test/preprocess/files/test_BlinkTask.c
C:/Users/davidfallah/Documents/IAR Projects/blinky/vendor/ceedling/lib/ceedling/preprocessinator_extractor.rb:18:in `readlines'
C:/Users/davidfallah/Documents/IAR Projects/blinky/vendor/ceedling/lib/ceedling/preprocessinator_extractor.rb:18:in `extract_base_file_from_preprocessed_expansion'
C:/Users/davidfallah/Documents/IAR Projects/blinky/vendor/ceedling/lib/ceedling/preprocessinator_file_handler.rb:14:in `preprocess_file'
C:/Users/davidfallah/Documents/IAR Projects/blinky/vendor/ceedling/lib/ceedling/preprocessinator.rb:40:in `preprocess_file'
C:/Users/davidfallah/Documents/IAR Projects/blinky/vendor/ceedling/lib/ceedling/preprocessinator.rb:12:in `block in setup'
C:/Users/davidfallah/Documents/IAR Projects/blinky/vendor/ceedling/lib/ceedling/preprocessinator_helper.rb:33:in `preprocess_test_file'
C:/Users/davidfallah/Documents/IAR Projects/blinky/vendor/ceedling/lib/ceedling/preprocessinator.rb:25:in `preprocess_test_and_invoke_test_mocks'
C:/Users/davidfallah/Documents/IAR Projects/blinky/vendor/ceedling/lib/ceedling/test_invoker.rb:42:in `block in setup_and_invoke'
C:/Users/davidfallah/Documents/IAR Projects/blinky/vendor/ceedling/lib/ceedling/test_invoker.rb:32:in `setup_and_invoke'
C:/Users/davidfallah/Documents/IAR Projects/blinky/vendor/ceedling/lib/ceedling/tasks_tests.rake:11:in `block (2 levels) in <top (required)>'
Tasks: TOP => default => test:all
(See full trace by running task with --trace)

--------------------
OVERALL TEST SUMMARY
--------------------

No tests executed.

I assume this is because the test file preprocessor should be copying test files under the build/test/preprocess/files directory, which currently doesn't happen.

After a bit of digging around I found this example configuration file for Unity that looks like it may be helpful. It's geared towards an IAR EW/Cortex M3 environment like the one I'm using. This may give some indication of what configuration options I need to specify in my Ceedling project.yml:

If I can get Ceedling to build and test the blinky project using an IAR toolchain, I'm hoping I can adapt it to work with my actual project. Any help would be appreciated.

Tagc
  • 8,736
  • 7
  • 61
  • 114
  • 1
    As far as I know, IAR linker is `ilinkarm`, `iccarm` is only the compiler. Compiler manual should give you the details on how to use them. Also, workbench has option to show full build commands on compile, which you could check for reference. – user694733 Dec 14 '16 at 10:41
  • @user694733 Thanks - I've made that change. Do you happen to know if `iccarm` is still appropriate for the `test_file_preprocessor` and `test_includes_preprocessor` roles? – Tagc Dec 14 '16 at 10:44
  • I haven't really compiled much from command line, so I cannot say for sure. And you should verify my earlier comment from the compiler manual: my IAR version is slightly older than yours so it may have changed. – user694733 Dec 14 '16 at 10:49
  • @user694733 Doing that now. I'm also looking over the full build logs from a complete rebuild of my actual project to see how `iccarm` and `ilinkarm` are being used - looks like `iccarm` has a `--preprocess` flag. – Tagc Dec 14 '16 at 10:50
  • Obviously the tests must be carried out on the target hardware or they won't be very meaningful. TDD was never meant to replace common sense. If you test them on a PC you can only detect pure algorithm problems and not problems with hardware, interrupts, memory, timing or poorly-specified behavior. If there is a requirement that the code should be portable, it would be wise to test it on multiple platforms. – Lundin Dec 14 '16 at 12:25
  • "If you test them on a PC you can only detect pure algorithm problems..." Which is only what I need for the time being. There is also no requirement for the code to be portable. – Tagc Dec 14 '16 at 12:26
  • To expand: purely algorithmic tests can be automated and performed on the host machine while testing on the actual hardware (to detect possible problems with timing, etc.) can be done manually (as is done at the moment). It's not obvious at all that tests must be carried out on target hardware to be meaningful, as James Grenning advocates a "dual-targeting" approach and claims both sorts of tests have value. – Tagc Dec 14 '16 at 12:32

2 Answers2

12

It was a struggle but I believe I've managed to configure Ceedling to help test my project. Hopefully this will be useful to anyone else looking to use Ceedling within IAR projects.

The Ceedling CLI has a command (ceedling new <proj_name>) that allows you to create new projects with the structure Ceedling expects. You can also specify the name of an existing project in which case it only adds the necessary files to make it Ceedling-compatible, which is what I did with my project.

For reference, my project structure looked something like this after performing this step:

.
├── build
│   ├── artifacts
│   │   └── test
│   ├── docs
│   ├── exe
│   ├── list
│   ├── logs
│   ├── obj
│   ├── temp
│   └── test
│       ├── cache
│       ├── dependencies
│       ├── list.i
│       ├── mocks
│       ├── out
│       ├── results
│       ├── runners
│       └── tests.map
├── project.yml
├── rakefile.rb
├── src
│   └── main
│       ├── c
│       │   ├── canDatabase.c
│       ├── include
│       │   ├── canDatabase.h
│       ├── python
│       └── resources
├── test
│   ├── support
│   └── test_canDatabase.c
├── <my_project>.dep
├── <my_project>.ewd
├── <my_project>.ewp
├── <my_project>.eww
├── vendor
│   └── ceedling
│       ├── docs
│       ├── lib
│       ├── plugins
│       └── vendor
└── version.properties

After that, I looked over the reference manuals for the IAR tools and studied the output from IAR Embedded Workbench when building sample projects, as @user694733 suggested. I used this information to edit my project.yml as given below:

:project:
  :use_exceptions: FALSE
  :use_test_preprocessor: FALSE
  :use_auxiliary_dependencies: TRUE
  :build_root: build
  :release_build: FALSE
  :test_file_prefix: test_

:environment:
  - :path:
    - 'C:\Program Files (x86)\IAR Systems\Embedded Workbench 6.5\arm\bin'
    - 'C:\Program Files (x86)\IAR Systems\Embedded Workbench 6.5\common\bin'
    - #{ENV['PATH']}

:extension:
  :executable: .out

:paths:
  :test:
    - +:test/**
    - -:test/support
  :source:
    - src/main/c/**
    - src/main/include/**
    - src/main/resources/**
  :support:
    - test/support

:defines:
  :commmon: &common_defines []
  :test:
    - *common_defines
    - TEST
  :test_preprocess:
    - *common_defines
    - TEST

:cmock:
  :mock_prefix: mock_
  :when_no_prototypes: :warn
  :enforce_strict_ordering: TRUE
  :plugins:
    - :ignore
    - :callback
  :treat_as:
    uint8:    HEX8
    uint16:   HEX16
    uint32:   UINT32
    int8:     INT8
    bool:     UINT8

:tools:
  :test_compiler:
    :executable: iccarm
    :name: 'IAR test compiler'
    :arguments:
      - -D _DLIB_FILE_DESCRIPTOR=1
      - --debug
      - --endian=little
      - --cpu=Cortex-M3
      - -e
      - --fpu=None
      - -Ol
      - --preprocess "build/test/list"
      - --dlib_config "C:/Program Files (x86)/IAR Systems/Embedded Workbench 6.5/arm/INC/c/DLib_Config_Normal.h"
      - -I"$": COLLECTION_PATHS_TEST_TOOLCHAIN_INCLUDE
      - -I"$": COLLECTION_PATHS_TEST_SUPPORT_SOURCE_INCLUDE_VENDOR
      - -o "${2}"
      - --diag_suppress=Pa050
      - '"${1}"'

  :test_linker:
    :executable: ilinkarm
    :name: 'IAR test linker'
    :arguments:
      - --vfe
      - --redirect _Printf=_PrintfFull
      - --redirect _Scanf=_ScanfFull
      - --semihosting
      - --config "C:/Program Files (x86)/IAR Systems/Embedded Workbench 6.5/arm/config/generic_cortex.icf"
      - --map "build/test/tests.map"
      - -o "${2}"
      - '"${1}"'

  :test_fixture:
    :executable: cspybat
    :name: 'CSpyBat test runner'
    :arguments:
      - '"C:\Program Files (x86)\IAR Systems\Embedded Workbench 6.5\arm\bin\armproc.dll"'
      - '"C:\Program Files (x86)\IAR Systems\Embedded Workbench 6.5\arm\bin\armsim2.dll"'
      - '"${1}"'
      - --plugin "C:\Program Files (x86)\IAR Systems\Embedded Workbench 6.5\arm\bin\armbat.dll"
      - --backend -B
      - --endian=little
      - --cpu=Cortex-M3
      - --fpu=None
      - --semihosting

:plugins:
  :load_paths:
    - vendor/ceedling/plugins
  :enabled:
    - stdout_pretty_tests_report
    - module_generator
...

This seems to be a suitable configuration for testing code designed to work on a Cortex-M3 device.

I also edited rakefile.rb to ensure that the generated test files are cleaned before each test run, as this was necessary to have the test results get printed consistently.

PROJECT_CEEDLING_ROOT = "vendor/ceedling"
load "#{PROJECT_CEEDLING_ROOT}/lib/ceedling.rb"

Ceedling.load_project

task :default => %w[ clean test:all ]

I was then able to define and run unit tests. Below is an excerpt from test_canDatabase.c:

#include "unity.h"
#include "canDatabase.h"

uint32_t actualId;
uint8_t actualPayload[8];
uint8_t actualPayloadLen;
uint8_t actualCanPort;

void mockHandler(uint32_t id, uint8_t payload[8], uint8_t payloadLen, uint8_t canPort)
{
  actualId = id;
  actualPayloadLen = payloadLen;
  actualCanPort = canPort;

  for (int i=0; i < payloadLen; i++)
  {
    actualPayload[i] = payload[i];
  }
}

void setUp(void)
{
  actualId = 0;
  actualPayloadLen = 0;
  actualCanPort = 0;

  for (int i=0; i < 8; i++)
  {
    actualPayload[i] = 0;
  }

  CANDB_Init(mockHandler);
}

void tearDown(void) {}

void test_Register_Tx_Definition()
{
  // GIVEN a CAN Tx message definition.
  CAN_TX_MESSAGE_DEFINITION_T definition;
  definition.id = 0;

  // WHEN we register the definition in the CAN database.
  int err = CANDB_RegisterTxDefinition(definition);

  // THEN the database should return SUCCESS (0x0).
  TEST_ASSERT_EQUAL_MESSAGE(0x0, err, "Registration should succeed");
}

void test_Register_Tx_Definition_Twice()
{
  // GIVEN a CAN Tx message definition.
  CAN_TX_MESSAGE_DEFINITION_T definition;
  definition.id = 0;

  // WHEN we register the definition once.
  CANDB_RegisterTxDefinition(definition);

  // AND we register the definition again.
  int err = CANDB_RegisterTxDefinition(definition);

  // THEN the database should return SUCCESS (0x0).
  TEST_ASSERT_EQUAL_MESSAGE(0x0, err, "Re-registration should succeed");
}

I'm now able to run automated tests by invoking "ceedling" from a terminal (project root is the current working directory):

$ ceedling
---------------------
BUILD FAILURE SUMMARY
---------------------
Unit test failures.


Cleaning build artifacts...
(For large projects, this task may take a long time to complete)



Test 'test_canDatabase.c'
-------------------------
Generating runner for test_canDatabase.c...
Compiling test_canDatabase_runner.c...
Compiling test_canDatabase.c...
Compiling unity.c...
Compiling canDatabase.c...
Compiling cmock.c...
Linking test_canDatabase.out...
Running test_canDatabase.out...

-----------
TEST OUTPUT
-----------
[test_canDatabase.c]
  - ""
  - "     IAR C-SPY Command Line Utility V6.6.0.2752"
  - "     Copyright 2000-2013 IAR Systems AB."
  - ""
  - ""

-------------------
FAILED TEST SUMMARY
-------------------
[test_canDatabase.c]
  Test: test_Register_More_Than_Max_Allowed_Definitions
  At line (84): "Expected 1 Was 0. Registration > CANDB_MAX_TX_DEFINITIONS should fail"

  Test: test_Activate_Tx_Definition_With_Hardcoded_Payload
  At line (124): "Expected 0x00000001 Was 0x00000000. Incorrect ID"

--------------------
OVERALL TEST SUMMARY
--------------------
TESTED:  4
PASSED:  2
FAILED:  2
IGNORED: 0
Tagc
  • 8,736
  • 7
  • 61
  • 114
2

Your answer was super helpful and I'm working on similar approach for Cortex-M4 with IAR 7.5 and thought of sharing my portions of project.yml additions.

:tools:
  :test_compiler:
    :executable: iccarm
    :name: 'IAR test compiler'
    :arguments:
      - -D _DLIB_FILE_DESCRIPTOR=1 
      - --diag_suppress Pa050,Pa082,Pa039,Pe186
      - --no_cse
      - --no_unroll
      - --no_inline
      - --no_code_motion
      - --no_tbaa
      - --no_clustering
      - --no_scheduling
      - --debug
      - --endian=little
      - --cpu=Cortex-M4
      - -e
      - --fpu=VFPv4_sp
      - --dlib_config "C:\Program Files (x86)\IAR Systems\Embedded Workbench 7.5\arm\INC\c\DLib_Config_Normal.h"
      - --preprocess "build/test/list"
      - -On
      - --c++
      - --no_exceptions
      - --no_rtti
      - -I"$": COLLECTION_PATHS_TEST_TOOLCHAIN_INCLUDE
      - -I"$": COLLECTION_PATHS_TEST_SUPPORT_SOURCE_INCLUDE_VENDOR
      - -o "${2}"
      - '"${1}"'

  :test_linker:
    :executable: ilinkarm
    :name: 'IAR test linker'
    :arguments:
      - --vfe
      - --redirect _Printf=_PrintfFull
      - --redirect _Scanf=_ScanfFull
      - --semihosting
      - --keep __checksum
      - --entry __iar_program_start
      - --place_holder __checksum,4,.checksum,4
      - --define_symbol __checksum_begin=0x8020000
      - --define_symbol __checksum_end=0x80dfffb
      - --no_exceptions 
      - --config "C:\Users\jseinfeld\RubymineProjects\m4\common\cb\proj\flash.icf"
      - --map "build/test/m4.map"
      - -o "${2}"
      - '"${1}"'

  :test_fixture:
    :executable: cspybat
    :name: 'CSpyBat test runner'
    :arguments:
      - '"C:\Program Files (x86)\IAR Systems\Embedded Workbench 7.5\arm\bin\armproc.dll"'
      - '"C:\Program Files (x86)\IAR Systems\Embedded Workbench 7.5\arm\bin\armsim2.dll"'
      - '"${1}"'
      - --plugin "C:\Program Files (x86)\IAR Systems\Embedded Workbench 7.5\arm\bin\armbat.dll"
      - --backend -B
      - --endian=little
      - --cpu=Cortex-M4
      - --fpu=VFPv4_sp
      - --semihosting
Cac3a
  • 117
  • 1
  • 10