wrong evaluation of the compiler??... Very rare error

 wrong evaluation of the compiler??... Very rare error

Dear all,

I have this ultra simple strcture

typedef struct
{ uint8_t Hora; uint16_t Data1;
} EEpromPaqueteDatosS;

When I do sizeof(EEpromPaqueteDatosS) it returns 4!!! But if I do the sizeof of the same structure with only one variable, in other words, with uint8_t variable it returns 1 and with uint16_t it returns 2.. but if the structure have the two variables it returns 4!!

Also if I copy the structure to a uint8_t vector I can found a strange byte in the middle.. like this:

EEpromPaqueteDatosS EEpromPaqueteDatos;
uint8 data[4];

EEpromPaqueteDatos.Hora = 0x10;
EEpromPaqueteDatos.Data1= 0x1020;

When I copy the structure to data, data is like 0x10, 0x??, 0x20, 0x10.. why this extra byte!!!!

Can anybody help me with this???

Thanks

 RE: wrong evaluation of the compiler??... Very rare error

You just disovered the padding phenomenon! It's perfectly normal. Any book on the C programming language should have this topic covered.

 RE: wrong evaluation of the compiler??... Very rare error

Ok, sorry for my ignorance.. can you explain it to me and givem e a solution?

 RE: wrong evaluation of the compiler??... Very rare error

Yes - the normal solution is to let the compiler do what the compiler does.

It's common that data types larger than one byte have a requirement on alignment. So two-byte integers must often be stored on even addresses.

And the compiler must then also make sure that the full struct gets the same align as that internal element. So it doesn't matter if you have 16-bit + 8-bit fields - the compiler still needs to make the struct have same align as the 16-bit field. Hence if processor requires align for 16-bit field, then the full struct must have a size that is 2*n so that you can have an array of structs and each and every struct will start on a 2*n address.

Most compilers have some option to pack structs, but that is costly. If the processor do need 16-bit align for a 16-bit number than a packed structure requires the compiler to produce code that picks up the field as two 8-bit reads and then combines them. Ugly and costly.

So you normally do not want to play with packed data unless you really, really, really do need to. All covered on the net. Google has lots and lots of info about this. This forum too.

 RE: wrong evaluation of the compiler??... Very rare error

#pragma pack(1)

This is the solution... don't know this.

Thanksss!!!!!

 RE: wrong evaluation of the compiler??... Very rare error

Thanks for your explanation.

I need a very short strcture to save in external I2C eeprom the maximum I can... can let the compiler to add more bytes.

Thanks!!

 RE: wrong evaluation of the compiler??... Very rare error

Using __packed is probably better than using #pragma pack since __packed "sticks" to the type of the members but there are cases where the alignment info can get lost when using #pragma pack.

 when donosaurs roamed the earth ...

when donosaurs roamed the earth, memory was scarce and costly.

today, memory is a commodity.

if you decide to pack your structures they will take less memory (who cares) but have a speed penalty (you should care).

sure, the above, as all generalization, will have exceptions.

anyhow, a problem I ran into that deserves mentioning:

If you have a structure that start with a char, the packings inserted may vary based on where ste struct end up being linked. This is a non-problem, unless you copy the struct.

Erik

 RE: when donosaurs roamed the earth ...

> if you decide to pack your structures they will take less memory
> (who cares) but have a speed penalty (you should care).

It depends. Many smartcards, intelligent sensors, etc. are using ARM
processors these days and offer only some hundred byte of RAM, while
the processing performance leaves enough headroom to justify less
efficient memory access.

> If you have a structure that start with a char, the packings
> inserted may vary based on where ste struct end up being linked.

I'd be surprised, since the location of struct members must be known
at compile time.

Kind regards,
Marcus
http://www.doulos.com/arm/

 RE: when donosaurs roamed the earth ...

Some programs are code-space limited.
Some are RAM-space limited.

Most of the time, people don't reach any of the limits.

Sometimes, the packed data get you into extra troubles because multiple memory accesses means a read of the value can't be performed atomically.

So it isn't something to do unless there is a very good motive for it.

 RE: when donosaurs roamed the earth ...

My post included:
"sure, the above, as all generalization, will have exceptions."

Which should have covered It depends

this
>
> If you have a structure that start with a char, the packings
> inserted may vary based on where ste struct end up being linked.

I'd be surprised, since the location of struct members must be known
at compile time.
example
struct
U8 x
U16 y

if the linker happen to be at an odd address when it link the struct no packing will happen, if it is at an even address there will be a packbyte inserted between the two.

I solved the problem this way
struct
U32 dummy
U8 x
U16 y

Sometimes, the packed data get you into extra troubles because multiple memory accesses means a read of the value can't be performed atomically.
I could see some processors burp on this. I do not know if any ARMs would, but a good argument for not packing

Erik

 RE: when donosaurs roamed the earth ...

Basically, processors who can read "odd-aligned" data can do it because they have an intelligent memory controller that puts the processor core to sleep while the memory controller performs multiple memory accesses and then merges the relevant bytes.

The memory controllers inside Intel x86 chips have a tradition of silently hiding odd aligns. So x86 programs have by tradition always managed, while same source code recompiled on Sun or other platforms have resulted in "bus error" exceptions when the memory controller have received an unsupported request.

For processors with this feature in the memory controller interface, you have compiler options to specify align just for performance. Even if the memory interface is pipelined, you still increase the probability of a pipeline stall when the memory controller must perform two sequential accesses.

For processors where the memory controller can not perform this merge, the compiler really must insert code for multiple reads/writes. A 4-byte integer one byte off can requires a read/modify/write to fix that odd byte and a read/modify/write to fix the other three bytes. Few processors have a lock primitive where the processor can lock the bus (dma or possibly external memory accesses by other devices) and/or lock interrupts from interfere with these two read/modify/write actions.

About structs and linking - a "normal" compiler/linker, i.e. a compiler/linker that are following the C language standard, would not do any three-byte optimization of a struct just because the linker has the option to place the struct at an odd or even location. If the struct has an alignment critieria larger than 1, then the compiler has to make the struct n times that alignment criteria. And make the relevant fields inside the struct placed so they fulfill that requirement. And the linker must place the full struct so the struct is aligned to this - or better - alignment level.

So there isn't a difference between:

struct {
    uint8_t a;
    uint16_t b;
} alt_1;

struct {
    uint16_t a;
    uint8_t b;
} alt_2;

If processor needs to have uint16_t aligned to even address, then alt_1 must be n*2 bytes large with field b at an even offset inside the struct. And alt_2 must be n*2 bytes large with a at an even offset inside the struct.

So sizeof(alt_1) and sizeof(alt_2) will both be n*2 bytes large. And a memcpy(&alt_1,...) or memcpy(&alt_2,...) will be able to perform a memory copy without reacing any byte outside of the struct.

And it will be possible to create arrays of above types and ever element in that array will start at a n*2 byte address since each element will be n*2 bytes large.

Note that the C51 compiler don't need to worry about align from a processor perspective since the 8051 performs 8-bit memory accesses. So it's ok for the compiler to use zero padding and make above structures 3 bytes large. And the linker will be free to place variables of these types at odd or even addresses without needing to worry.

The memory controllers in most ARM chips will not like the above as 3 byte structures, so the compiler will either add a pad or add a number of extra instructions to make only aligned accesses.

 RE: when donosaurs roamed the earth ...

If the linker happen to be at an odd address when it link the struct no packing will happen, if it is at an even address there will be a packbyte inserted between the two.

No, it won't. It's forbidden by the language definition.

A given struct will always have the exact same layout, no matter where it ends up: on the stack, on the heap, or in statically allocated memory --- of which the latter is the only one the linker is really even involved with in any way.

 RE: when donosaurs roamed the earth ...

If the linker happen to be at an odd address when it link the struct no packing will happen, if it is at an even address there will be a packbyte inserted between the two.

No, it won't. It's forbidden by the language definition.

well, if so Keil (recent version) breaks the "language definition"

I caught this as a bug, and fixed it instead oof complaining to Keil.

I do not know the "language definition" but the case in point was a const struct for the defaults and a struct for the initialized values. copying the defaults into the variable struct blew the thing.

Erik

 RE: when donosaurs roamed the earth ...

"I do not know the "language definition" but the case in point was a const struct for the defaults and a struct for the initialized values. copying the defaults into the variable struct blew the thing."

Post the code sequence, and we may be able to come up with a reason for the problem.

 RE: when donosaurs roamed the earth ...

if so Keil (recent version) breaks the "language definition"

I'm with Per on that one: it'll take actual evidence to convince me of that. In a new thread, preferrably.

I do not know the "language definition"

And you say that (here and elsewhere) as if you're proud of it. I think you rather should be ashamed of it.

 RE: wrong evaluation of the compiler??... Very rare error

#pragma pack(1)

This is the solution.

No, it is not "the" solution. It is what many people believe to be the only solution. Unfortunately, many of those people are wrong a good portion of the times. What these people ignore are the problem this so-called solution creates by itself.

The central issue though is that the problem this is supposed to solve typically isn't actually a problem at all. And even if does happen to be an actual problem in a given case, packing the structure may easily create more problems than it solves.

 RE: packing the structure may easily create more problems than it solves.

Been There. Done That.

It might seem like a good idea to try to match your comms protocol packets to packed 'C' structures like this - but it can, indeed, end up as more trouble than it's worth.

Bet to create functions to "serialise" and "de-serialise" your data: http://en.wikipedia.org/wiki/Serialization

http://www.lmgtfy.com?q=serialise+deserialise

 RE: packing the structure may easily create more problems than it solves.

I never pack a structure intended for the cable or similar. Always an encode/decode or pack/unpack or serialize/unserialize explicitly implemented. It is 100% visible what happens, so you don't need to wonder when debugging. And the code can be moved to another processor with different byte order and the pack/unpack will still do as intended. So less code to rewrite when supporting multiple architectures.

 RE: wrong evaluation of the compiler??... Very rare error

Can anybody help me with this???

You could start by helping yourself in various ways.

1) Try to take your own writing seriously, e.g. by finishing sentences with an appropriate amount of punctuation. That usually means one full stop, not half a dozen exclamation or question marks.

2) Stop blaming your tools all that easily. You need to get to grips with the idea that lots of people have been using these tools intensely before you started, so when you thing you have a problem with them, it's quite extremely unlikely that the problem really is on the tools' end of things.

3) Get some proper training, or at the very least a good textbook. You're developing a habit of jumping to conclusions based on incorrect beliefs. Wherever those beliefs come from, you need to replace that source of information by a better one.

 RE: Get some proper training

Absolutely!!

"Many people seem to adopt the jump-in-at-the-deep-end-and-hope-you-learn-to-swim-before-you-drown approach to start developing in 'C'"
http://blog.antronics.co.uk/2011/08/08/

 RE: wrong evaluation of the compiler??... Very rare error

Hi,

Hans, you are right, but, don't think my phrase was as bad as you said.. but, I will try to ask the questions better.

By the other hand memory is not important today, but, in some cases 3 or 4 more bytes when all data is 10 bytes in cost sensitive applications can be a problem.

Thanks.

 RE: wrong evaluation of the compiler??... Very rare error

"By the other hand memory is not important today, but, in some cases 3 or 4 more bytes when all data is 10 bytes in cost sensitive applications can be a problem."

You are not likely to find a chip with only 10 bytes of data, so you will then not get 30% or 40% of filler bytes. So a single struct can be used without pack.

And if you need to store 1000 structures, you have enough that it is meaningful to see it as a database and create a "read_record()" and "wrtie_record()" that handles pack/unpack.

 RE: wrong evaluation of the compiler??... Very rare error

Hi,

When you have to store 10 bytes every 1 hour and the program must write data for some month, 3 or 4 bytes is very important, is the difference berween store 40 days or 70 days using a memory lime M24C128... You can place a larger memory or two memories, but, when is a cost sensitive application, you cant...

 RE: When you have to store 10 bytes every 1 hour and the program must write data for some month

In that case, as already discussed, you should not be writing structures - you should have read() and write() functions to access the storage as a "stream"...

 RE: wrong evaluation of the compiler??... Very rare error

If I write a data logger, where memory is important, I would probably even check if I need 8 + 16 bits or maybe just 5 + 12 bits.

I might even, depending on type of sample, see if I can have one bit of data represent if following record is having absolute or relative values. Storing temperatures as deltas can save a lot of memory space.

But trying to use raw C structs for binary storage on a memory media is just plain silly. It's a great example of walking the wrong path. Especially so if that memory is removeable and should potentially be read by another device.

The language don't have a standard for packing structures just because it isn't a good way to solve the kind of problem you are trying to solve.

Time to take two steps back and switch to a real solution. A solution that is not depending on compiler flags, pragmas or other tricks.

 RE: wrong evaluation of the compiler??... Very rare error

By the way - your 8-bit value is named "Hora". That would be "hour". If your logger do record one sample every hour, you don't have a need to store any hour value in the sample. So 3 bytes suddenly became 2 bytes. And if the 16-bit value can never be larger than what fits in 15 bits, you could use one bit as "flag" to store a hour jump in case the device has been turned off and there is a jump in hours since previous sample recorded.

In the end, your packed data would probably waste a lot of memory, if memory really are important.

 RE: wrong evaluation of the compiler??... Very rare error

This was an example to solve the problem, in my case the sturcture is more complex and hard to explain... Is curious that the people try to say absolute solutions without knowing at all the question like use "write" o r "read" like stream or other "solutions" posted here without know other things related to the device or its operation... The problem is solved and this discusion have no sense now, anyway, thaks.

 had I started this from scratch I would have ...

had I started this from scratch I would have created the struct the way i have done ever sice my first exposure to an "alignment critical" processor

I always make structs this way:
dwords
words
bytes

that way there never is an alignment issue

Erik

 RE: without know other things related to the device or its operation

Since you didn't give any of those details, how could anyone else know them?!

Therefore, we discuss general principles - that can be applied in all cases.

"this discusion have no sense now"

Note that this is not a free consultancy service - it is an open discussion forum.

It seems that there are still issues worthy of discussion - even if not specifically pertinent to your particular application at this time.

 RE: wrong evaluation of the compiler??... Very rare error

"Is curious that the people try to say absolute solutions without knowing at all the question like use "write" o r "read" like stream or other "solutions" posted here without know other things related to the device or its operation.."

It is "curious" that the compiler must implement code to serialize read and write if you play with a packed structure. Which is a reason why the general recommendation is to not use a compiler-specific attribute to hide an issue in a way that fools other developers and can result in huge problems with concurrency etc.

It is "curious" that the normal solution is to explicitly call code that does the pack/unpack, where what happens is visible and can be debugged or potentially optimized depending on the specific needs.

That you did write the answer you did, indicates that you did not spend some time really thinking through the answers you have received. If you had, and didn't agree with some of it, you would have posted specific follow-up questions or "wouldn't this ..." comments instead.

Well, you are free to run into walls if that makes you happy. We don't care if you use problematic structures. We can suggest reasons to avoid doing it, but it is still up to you to ignore recommendations if you think maybe 3-digit years of experience isn't relevant.

 the usual reason for this is

Is curious that the people try to say absolute solutions without knowing at all the question like use "write" o r "read" like stream or other "solutions" posted here without know other things related to the device or its operation..

the usual reason for this is that the question was formulated in a vague way

Erik

 RE: wrong evaluation of the compiler??... Very rare error

The problem is solved

No, it's not. You only believe it is, solely based on picking exactly the wrong pieces of information out of the wealth you've been given.

and this discusion have no sense now,

You're wrong about that. Just because you refuse to learn something doesn't mean others can't.