Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Question: library and two executable. stm libs are compiled 3 times? #195

Open
andyinno opened this issue May 10, 2021 · 14 comments
Open

Question: library and two executable. stm libs are compiled 3 times? #195

andyinno opened this issue May 10, 2021 · 14 comments

Comments

@andyinno
Copy link

andyinno commented May 10, 2021

Following this example:

if I have lib foo with common code:
and I link lib foo with:
HAL::STM32::F4::GPIO
and I use this lib with executables bar1 and bar2 linking foo.

I see in the output that lib gpio is compiled 3 times. I was expecting that gpio lib is build only once.

Am I missing something?

This is just an example but basically what I am trying to do is to have some libraries with a lot of common code that compile different executables.

This is happening becaue I think I am forced to add:
target_link_libraries(mygpiolib PUBLIC HAL::STM32::F4::GPIO )

If I make the mygpiolib PRIVATE, then the executable is not able to find the GPIO define and include file because it is not inherited. If I add the HAL::STM32::F4::GPIO to the executable, then it is in any case built twice.

Am I missing something stupid here?
thank you in advance

@atsju
Copy link
Collaborator

atsju commented May 10, 2021

Hello @andyinno
If a library is used multiple times it's files should be compiled only once and then the library must be linked several times exactly as you expect.

I don't think this question is directly related to this repository https://github.com/ObKo/stm32-cmake but more like a cmake usage problem.
However, if you feel like this is the good place to ask for help or if there is a bug in how we create libraries it would be easier if you could provide a link to a repository showing clearly the problem.
If the example you provide is a fork of this repository and is derived from one of the existing examples then it's much more easy to help.

@Hish15
Copy link
Collaborator

Hish15 commented May 10, 2021

Hi @andyinno,

In this repo INTERFACE libraries are used to define all the libraries, so quoting the documentation :
an INTERFACE library target does not directly create build output, though it may have properties set on it...

you will build HAL::STM32::F4::GPIO as many times as you link to it directly.

What can be done is define a static library myStatic, link GPIO to it (depending on you cmake version you will need to add some source code to this library), and then link all your executables to this target (myStatic), then you will have a single build for files related to GPIO.

@andyinno
Copy link
Author

I don't think this question is directly related to this repository...

First of all, thank you for the reply.
Well... I think it is because examples shows how to use the different modules. And they only explain how to link to executables.
What I am observing is different from what you are explaining. When adding the library, every source file is transitively compiled multiple times.

What can be done is define a static library myStatic, link GPIO to it (depending on you cmake version you will need to add some source code to this library), and then link all your executables to this target (myStatic), then you will have a single build for files related to GPIO.

Yes, that was my workaround for it. I was just unsure if what I was passing to cmake was correct or not.
It seemed strange to me that I did not found any reference to it in the entire project. I think the library reuse is a common pattern and having a dependency build a lot of times is time consuming for big projects.

@andyinno
Copy link
Author

@Hish15 : If I am not wrong, this is what you are suggesting.

https://github.com/andyinno/stm32-cmake/tree/blinky_lib here I created an example lib, blinky_lib. Trying to compile it, only for F4 devices...

output from compilation is:
`
cmake -DCMAKE_TOOLCHAIN_FILE=../../cmake/stm32_clang.cmake ..
make
[ 0%] Generating F407VG.ld
[ 0%] Built target CMSIS_LD_F407VG
[ 20%] Building C object CMakeFiles/toggle-f4.dir/pin_toggle.c.obj
[ 20%] Building C object CMakeFiles/toggle-f4.dir/opt/STM32CubeF4/Drivers/STM32F4xx_HAL_Driver/Src/stm32f4xx_hal_rcc.c.obj
[ 20%] Building C object CMakeFiles/toggle-f4.dir/opt/STM32CubeF4/Drivers/STM32F4xx_HAL_Driver/Src/stm32f4xx_hal.c.obj
[ 20%] Building C object CMakeFiles/toggle-f4.dir/opt/STM32CubeF4/Drivers/CMSIS/Device/ST/STM32F4xx/Source/Templates/system_stm32f4xx.c.obj
[ 40%] Building C object CMakeFiles/toggle-f4.dir/opt/STM32CubeF4/Drivers/STM32F4xx_HAL_Driver/Src/stm32f4xx_hal_gpio.c.obj
[ 40%] Building C object CMakeFiles/toggle-f4.dir/opt/STM32CubeF4/Drivers/STM32F4xx_HAL_Driver/Src/stm32f4xx_hal_cortex.c.obj
[ 40%] Building ASM object CMakeFiles/toggle-f4.dir/opt/STM32CubeF4/Drivers/CMSIS/Device/ST/STM32F4xx/Source/Templates/gcc/startup_stm32f407xx.s.obj
[ 60%] Linking C static library libtoggle-f4.a
[ 60%] Built target toggle-f4
[ 60%] Building C object CMakeFiles/stm32-blinky-f4.dir/blinky.c.obj
[ 80%] Building C object CMakeFiles/stm32-blinky-f4.dir/opt/STM32CubeF4/Drivers/STM32F4xx_HAL_Driver/Src/stm32f4xx_hal_rcc.c.obj
[ 80%] Building C object CMakeFiles/stm32-blinky-f4.dir/opt/STM32CubeF4/Drivers/STM32F4xx_HAL_Driver/Src/stm32f4xx_hal.c.obj
[ 80%] Building C object CMakeFiles/stm32-blinky-f4.dir/opt/STM32CubeF4/Drivers/CMSIS/Device/ST/STM32F4xx/Source/Templates/system_stm32f4xx.c.obj
[ 80%] Building C object CMakeFiles/stm32-blinky-f4.dir/opt/STM32CubeF4/Drivers/STM32F4xx_HAL_Driver/Src/stm32f4xx_hal_gpio.c.obj
[100%] Building C object CMakeFiles/stm32-blinky-f4.dir/opt/STM32CubeF4/Drivers/STM32F4xx_HAL_Driver/Src/stm32f4xx_hal_cortex.c.obj
[100%] Building ASM object CMakeFiles/stm32-blinky-f4.dir/opt/STM32CubeF4/Drivers/CMSIS/Device/ST/STM32F4xx/Source/Templates/gcc/startup_stm32f407xx.s.obj
[100%] Linking C executable stm32-blinky-f4.elf
[100%] Built target stm32-blinky-f4

`

Each library C file is compiled twice, once for the library, once for the executable.

Is my understanding correct? This is what you meant?

@Hish15
Copy link
Collaborator

Hish15 commented May 13, 2021

Hi @andyinno,

After some testing (cmake 3.16) it seems that I was wrong :

Adding sources to the interface target, populate INTERFACE_SOURCES property. linking against this interface will propagate this property to other targets too. So you will have to compile multiple times as you noticed...

The only way to avoid this behavior in your example is to make the target toggle-f4 an interface

add_library(toggle-f4 INTERFACE )

target_link_libraries(toggle-f4 INTERFACE
    HAL::STM32::F4::RCC
    HAL::STM32::F4::GPIO
    HAL::STM32::F4::CORTEX
    CMSIS::STM32::F407VG
    STM32::NoSys)
target_sources(toggle-f4 INTERFACE pin_toggle.c)

This postpone the problem, but works in your example. You will have a probleme only when a real output will be need from a target (STATIC/OBJECT/EXECUTABLE)

I have no solution on how to avoid that cleanly, but I will think about it.

@atsju
Copy link
Collaborator

atsju commented May 13, 2021

Hi @andyinno
First of all @Hish15 answer is better than mine. Its correct that files will build as many times as they are included and the solution of static library should be working in this case.
Also thank you for the fork which permits to exchange on an identical code basis.

I built you blinky lib using VScode on windows with GCC and cmake 3.20.2
andyinno@9999298
Ans this is the output:

[main] Building folder: blinky_lib 
[build] Starting build
[proc] Executing command: "C:\Program Files\CMake\bin\cmake.exe" --build c:/GIT/stm32-cmake_andyinno/examples/blinky_lib/build --config Debug --target all -j 18 --
[build] [1/17   5% :: 0.043] Generating F407VG.ld
[build] [15/17  11% :: 0.141] Building ASM object CMakeFiles/stm32-blinky-f4.dir/C_/GIT/STM32CubeF4/Drivers/CMSIS/Device/ST/STM32F4xx/Source/Templates/gcc/startup_stm32f407xx.s.obj
[build] [15/17  17% :: 0.152] Building ASM object CMakeFiles/toggle-f4.dir/C_/GIT/STM32CubeF4/Drivers/CMSIS/Device/ST/STM32F4xx/Source/Templates/gcc/startup_stm32f407xx.s.obj
[build] [15/17  23% :: 0.355] Building C object CMakeFiles/stm32-blinky-f4.dir/C_/GIT/STM32CubeF4/Drivers/CMSIS/Device/ST/STM32F4xx/Source/Templates/system_stm32f4xx.c.obj
[build] [15/17  29% :: 0.372] Building C object CMakeFiles/toggle-f4.dir/C_/GIT/STM32CubeF4/Drivers/CMSIS/Device/ST/STM32F4xx/Source/Templates/system_stm32f4xx.c.obj
[build] [15/17  35% :: 0.401] Building C object CMakeFiles/stm32-blinky-f4.dir/blinky.c.obj
[build] [15/17  41% :: 0.417] Building C object CMakeFiles/toggle-f4.dir/pin_toggle.c.obj
[build] [15/17  47% :: 0.424] Building C object CMakeFiles/toggle-f4.dir/C_/GIT/STM32CubeF4/Drivers/STM32F4xx_HAL_Driver/Src/stm32f4xx_hal.c.obj
[build] [15/17  52% :: 0.432] Building C object CMakeFiles/toggle-f4.dir/C_/GIT/STM32CubeF4/Drivers/STM32F4xx_HAL_Driver/Src/stm32f4xx_hal_cortex.c.obj
[build] [15/17  58% :: 0.434] Building C object CMakeFiles/stm32-blinky-f4.dir/C_/GIT/STM32CubeF4/Drivers/STM32F4xx_HAL_Driver/Src/stm32f4xx_hal.c.obj
[build] [15/17  64% :: 0.438] Building C object CMakeFiles/stm32-blinky-f4.dir/C_/GIT/STM32CubeF4/Drivers/STM32F4xx_HAL_Driver/Src/stm32f4xx_hal_gpio.c.obj
[build] [15/17  70% :: 0.439] Building C object CMakeFiles/toggle-f4.dir/C_/GIT/STM32CubeF4/Drivers/STM32F4xx_HAL_Driver/Src/stm32f4xx_hal_gpio.c.obj
[build] [15/17  76% :: 0.444] Building C object CMakeFiles/stm32-blinky-f4.dir/C_/GIT/STM32CubeF4/Drivers/STM32F4xx_HAL_Driver/Src/stm32f4xx_hal_cortex.c.obj
[build] [15/17  82% :: 0.469] Building C object CMakeFiles/toggle-f4.dir/C_/GIT/STM32CubeF4/Drivers/STM32F4xx_HAL_Driver/Src/stm32f4xx_hal_rcc.c.obj
[build] [16/17  88% :: 0.492] Building C object CMakeFiles/stm32-blinky-f4.dir/C_/GIT/STM32CubeF4/Drivers/STM32F4xx_HAL_Driver/Src/stm32f4xx_hal_rcc.c.obj
[build] [16/17  94% :: 0.559] Linking C static library libtoggle-f4.a
[build] [17/17 100% :: 0.794] Linking C executable stm32-blinky-f4.elf
[build] Build finished with exit code 0

I Just see the answer from @Hish15 while I'm writing this. And he has not same results as me so we will investigate further :)

@Hish15
Copy link
Collaborator

Hish15 commented May 13, 2021

@atsju and @andyinno,

The conclusion is switch to cmake 3.19 and your problem will disappear.
See release note 3.19 : Interface Libraries may now have source files added via add_library() or target_sources(). Those with sources will be generated as part of the build system.

Edit: it did work for @atsju not for me on ubuntu 20.04

@andyinno
Copy link
Author

First of all, thank you both for your tests.

I made a test right now with
$ cmake --version
cmake version 3.19.2

The output is the same as before. It still compiles the imported libraries multiple times.
I am sure that the cmake used during the compilation is the correct one because I added

cmake_minimum_required(VERSION 3.19)

at the top of my CMakeLists.txt

Could this be some bug specific to the linux build?

@atsju
Copy link
Collaborator

atsju commented May 14, 2021

Hi @andyinno . I'm pretty sure that @Hish15 built on linux but maybe share your complete configuration to double check (compiler, cmake, linux,...). I also noted that you use cmake -DCMAKE_TOOLCHAIN_FILE=../../cmake/stm32_clang.cmake ..
But in the commit cmakelist CMAKE_TOOLCHAIN_FILE is already defined to some gcc toolchain file for arm-none-eabi-gcc. Could you double check this ?

@ucayalifish
Copy link

Why do not use object libraries?

@Hish15
Copy link
Collaborator

Hish15 commented May 15, 2021

@ucayalifish, object library does not propagate dependencies:

cmake_minimum_required(VERSION 3.19)
project(foobar C)

add_library(bar OBJECT bar/bar.c)
target_sources(bar PRIVATE bar/bar.c)
target_include_directories(bar PUBLIC bar)

add_library(foo OBJECT)
target_sources(foo PRIVATE foo/foo.c)
target_include_directories(foo PUBLIC foo)
target_link_libraries(foo PUBLIC bar)

add_executable(foobar main.c)
target_link_libraries(foobar PRIVATE foo)

foo.c:(.text+0xe): undefined reference to 'bar'

@Hish15
Copy link
Collaborator

Hish15 commented May 15, 2021

@atsju, @andyinno

I did not tried with cmake-3.19 on my side on linux.
I did now, and the sources files are being built multiple times! only using INTERFACE targets all the way to the top avoids that.
And it does makes sense actually... having sources added as INTERFACE using target_sources(), tells cmake that any target linking to this one must handle the source file itself... So now the question is, who come it works on your side @atsju.

@dhoove
Copy link

dhoove commented Jan 8, 2023

I am also having this problem. When several library dependencies exist, the amount of unnecessary building becomes quite significant.

Does somebody have a simple project demonstrating a solution?

@yaqwsx
Copy link

yaqwsx commented Jan 16, 2024

@dhoove: The problem arises from the stupid design of the ST HAL libraries and I am afraid stm32-cmake cannot do much about it. Every header file in ST HAL includes stm32<family>xxx_hal_conf.h. The HAL expects it to be provided by the project. Thus, stm32-cmake is forced to treat HAL as INTERFACE library so the HAL headers can include the config file from your target.

The proper solution would be to ask ST to change the architecture. However, from my experience ST engineers seem to be quite unwilling to make changes and follow good programing practice.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

6 participants