why data compressor compress constants in Flash

I am using STM32F407
RAM start at 0x2000 0000
Flash start at 0x0800 0000 , size 1MB

I could see, that writing const only, shows that section has attribute RO.
Adding const volatile the section is market as RW and compression on the data is applied. What I do not understand, why are data, that are already on its place compressed. The structure that varible represent does not fit any more and its members contain incorrect data, since the decompression is not performed and even has no sense to be performed.

thank you for help.

Ludek

So maybe you could explain the concept of having volatile (asynchronously changeable) data in a non-modifyable flash address?

I think you are tricking yourself when you play with const volatile and then gives an absolute address in flash.

To do advanced stuff, it's better to use the scatter file and specify that a segment or object file should have data stored using a specific policy. And then make sure you give good load and execution regions to use.

I will give explanation.
I am using const in order to preven accidental write to the variables. I use const in order to be able to initialise variable. Because it is configuration that had to be present everytime, even without power.

I need to exchange this configuration via Ethernet. So if it is not volatile, some optimalisation can take place and use variable values as direct operands and the effect of configuration exchange will not take place.

That is why const volatile. And it has to be on a given address, because, it is in different flash bank allowing me to erase this part no affecting the program code. And finally, I am changing also firware remotely and configuration stay untouched.

I can live with compress off solution, but I am courious, why the data are compresed in this flash region. I gives me no sense, and even do not understand what linker planed to do with this compressed data in this place.

thank you for your patience,
Ludek

You are tricking the linker into thinking it is RAM data - because normal flash data isn't volatile, except when the flash block gets an erase command.

So the linker thinks it should have a compressed copy in flash (to save space) and extract to RAM on each reboot.

Next thing: volatile is about asynchronous changes. The compiler has access to very few registers. So just calling normal functions (assuming you don't inline them or bend your back backwards to get maximum global optimization) would normally be enough to avoid issues with caching. And many processors have dedicated instructions to force barriers.

Once more: consider using a scatter file.

It might depend how the load regions are nested, you'd need to show the scatter file.

Compression suggests it is being moved, I'd review the table Region$$Table$$Base to see what it is doing with the statics.

It would only need to be volatile if changed outside of normal program flow, ie interrupt, or hw changing registers in a peripheral. If you call another function to write flash that doesn't count, and it also won't save you from the flash being cached, as volatile will still read the stale content, at least potentially.

My system reboot after changing the flash configuration.
I need that no information from the configuration is stored as direct operands instead of using changed data from updated flash. So the volatile is I think appropriate. Is there any other method preventing it?

I still do not understand what compiler and linker plan to do with the compressed data. Where do they plan to decompress it. To the RAM? where. Or to the same location the compressed data are now, in Flash? From Flash to Flash in the same location?

I would like to learn how to use Scatter File, can you recommend me some good link?

thank you.
Ludek

Thanks for all suggestions,
Ok,

If I accept that compiler/linker behaviour is senseless because it does not know what to do with RW constant in flash, what can I do in order to meet my needs:

- I need the data to be constant - not changeable by standard code
- I need it to be placed in flash - in order to store configuration of the device
- I need to place it to a given address - so that it is in next erasable flash sector that main program
- I need it not to be optimized as I experienced once - I need that all the data of the big struct can be found at the given address and all its members.
- And even more, I need some starting values in my configuration stored.

During using it (over a year or two), I faced only two problems:
- once some data were stored to registers, so I added "volatile"
- some time the default data were compressed in my struct so I set "--compress off"
I did not needed scatter file or another approach to reach it.
But now, I am facing problem (described in other thread), so I started to ask "why" some things happen (compression of const data and so on) in order to more understand and probably to solve my issue (described in other thread).

I found, that this only fullfils "const volatile _AT_ = {default data}" . But it confuses a linker a bit... so I would like to do it really correct and clean.

Can scatter file help me with this? How?

thank you,
Ludek

You can slice the flash into an arbitrary number of individual address ranges with the scatter file.

So you can create a specific flash region that corresponds with a specific flash sector.

And you can configure in the scatter file that the RO data from a specific object file should be stored in this specific flash region.

Whe the scatter file has the same load and execution address range for this flash region, then the startup file will not contain any code to try to copy the data on boot - the loader knows the data is already at the correct location.

You want the data to be const - but it's being stored in flash that really makes the data write-protected.

I would like to see the code construct where the compiler did ignore reading the const data and instead hard-coded the values when generating the code. That should normally not happen, since the compiler doesn't even know the value of variables from a different object file, unless you really go all-out with global optimization.

If I accept that compiler/linker behaviour is senseless because it does not know what to do with RW constant in flash

That is SUCH a dumb thing to say!

Why the heck do you think a decent tool such as the compiler or linker should know specifics about the flash?

It's for the developer to determine what the flash should be used for, how it should be accessed and only then execute the requirements.

Per,

thank you for clear explanation. This could help I think.
Regarding the const...
The flash does not mean it is write protect from definition.
Simple write to the flash write to the flash, once it is unlocked and prepared for write.

And second, the write to the const variable is prevented by the compiler that stops you doing it.

Regarding the construct that fills the flash with something else I will write tomorrow my case.

have a nice evening.

Ludek

If you have a source file flash_config.h and flash_config.c:

flash_config.h:
=====
typedef struct { uint32_t version; uint32_t size; uint32_t my_info; ... uint8_t ... ...
} config_t;

extern const config_t config;
=====

flash_config.c:
=====
#include "flash_config.h"

const config_t config = { CONFIG_VERSION, sizeof(config_t), ...
};
=====

Then you have const data - so the linker will find RO data to store in flash.

The compiler sees the const keyword, and will complain about any attempt to write.

The compiler will only see the actual values when compiling flash_config.c, and will always need to perform memory accesses to pick up the actual values for code in other source files. So no need for any volatile - your flash reprogramming isn't really an asynchronous change. And no danger that the compiler will inline any value when generating code.

If you have code in the same source file, then you have to be aware that in classical C, const just means 'read-only'. But more modern C++ standards can use directly assigned const values as actual constant expressions.

const unsigned NUM_ELEMENTS = 10;
unsigned elements[NUM_ELEMENTS];

So NUM_ELEMENTS isn't a strict variable that happens to be read-only, that the produced code needs to read - it can be treated the same as:

enum {
    NUM_ELEMENTS = 10,
};
unsigned elements[NUM_ELEMENTS];

So with more recent C++ you need to think twice when you have:

const <datatype> <varname> = <value>;

and later in the same compilation unit have code that makes use of the <varname> variable. You shouldn't expect to be able to reprogram that flash memory address and store a different value into the variable - the produced code may have already treated the original value similar to a constexpr value.

Dear w tf, (thank you for suggestion, that I am who say what flash is... I like it.),
Dear Per,

Thank you for exhausting description about modern C++ and C in general. It helped me much. If I understand correctly what you tried to explain that my case could be solved by simply adding EXTERN to the HEADER, preventing variable optimalisation?
Volatile is reserved for prevention of accessing data partly when interrupt occurs, for example. I am not sure if I did not experienced the variable optimalisation even with extern (when not using volatila, just only const...) - but it has been already a year or two, hard to remember and hard to replicate, since the optimalisation is hard to force to repeat the same action. So I hope, extern will help and solve my needs, I want my code to be clear, clean, correct and me to learn and understand.

NEXT: I am sending the situation, that the data for initialisation of the constant are already stored in the constant destination. As you can see, the text COMPRESSED shows, that it is the load data, not the run time data...

--datacompressor off: Load Region LR$$.ARM.__AT_0x080E0000 (Base: 0x080e0000, Size: 0x00000400, Max: 0x00000400, ABSOLUTE)

Execution Region ER$$.ARM.__AT_0x080E0000 (Base: 0x080e0000, Size: 0x00000400, Max: 0x00000400, ABSOLUTE, UNINIT)

Base Addr Size Type Attr Idx E Section Name Object

0x080e0000 0x00000400 Data RW 311 .ARM.__AT_0x080E0000 dista_konfigurace.o

compressor on: Load Region LR$$.ARM.__AT_0x080E0000 (Base: 0x080e0000, Size: 0x00000400, Max: 0x00000400, ABSOLUTE, COMPRESSED[0x00000190])

Execution Region ER$$.ARM.__AT_0x080E0000 (Base: 0x080e0000, Size: 0x00000400, Max: 0x00000400, ABSOLUTE, UNINIT, COMPRESSED[0x00000190])

Base Addr Size Type Attr Idx E Section Name Object

0x080e0000 0x00000400 Data RW 311 .ARM.__AT_0x080E0000 dista_konfigurace.o

Construct is:
#define SECTOR_ADDR_for_Config 0x080E0000 //Flash starts at 0X08000000
const volatile DISTA_konfigurace_flash_typ F __attribute__((at(SECTOR_ADDR_for_Config)))=
{ { 0x00, 0x00, 0x00, 0x00,....

I am trying to understand, why this situation happens. I agree with w tf that I have some

knowledge gaps in code translation/linking basics. I have never needed, not even on ZX

Spectrum :-) .

Ludek

Note that your data is RW - which imply RAM. But the address represents flash. What should the linker do?

Make it into real RO data and then have the scatter file have one dedicated flash region for this data. Then there will be no copy and no compress.

And this most probably solves all your linking issues.

Absolute placement in the code works well to place a symbol on top of some hardware register/buffer. But the _compiler_ doesn't have access to your memory regions from the project file. So the compiler doesn't know the address is in flash.

The linker knows it's in flash but if the object file has already indicated RW you have forced the linker into a corner. The linker could fail. Or assume there is aliasing between readonly and writeonly memory - something that does happen sometimes for memory-mapped registers.

That's why the scatter file itself is so much more powerful than an absolute address specified in the source code.