How to make an interruptable code sequence in C?

 How to make an interruptable code sequence in C?

I am looking for opinions on how to create a code sequence that is written in "C" that must be performed as an uninterruptable sequence. I want to disable interrupts (globally) execute a code sequence and then re-enable interrupts. I am looking for your inputs as I don't see how to guarantee this from what I know about the "C" standard. I think the compiler is allowed to optimize the sequence so that the actual linear code could be placed outside my expected enable interrupt and disable interrupt sequence (start/end points). The compiler knows that the enable/disable of the interrupt is volatile and must be performed but it doesn't know that there is an architectural dependency to the code order I want. In other words it could be that part of my sequence gets optimized outside of where the interrupt is not globally disabled. As the opcode creation behavior is still correct but it is not from a system behavior point of view.

Outside of writing it in assembly has anyone experienced this and how did you end up handling it. Thanks in advance for your inputs.

 RE: How to make an interruptable code sequence in C?

Outside of writing it in assembly has anyone experienced this and how did you end up handling it.

Depending on the compiler you use, it may already offer facilities to make parts of the code or whole functions uninterruptible.

If you use an operating system, it may offer such facilities, too.

If you are really worried about wild compiler optimizations resulting in things being done completely out of the intended sequence, there's always the option of checking the generated assembly code for conformity to the expectations and adjusting the optimization level if the instructions are not in the correct order. Usually, the compiler manual will state which optimizations are permitted at which optimization level.

There are also other options, like using a "wrapper" function written in assembly which takes care of disabling interrupts, calls the C function that is supposed to be uninterruptible, and then restores the interrupt state.

 RE: How to make an interruptable code sequence in C?

I think the compiler is allowed to optimize the sequence so that the actual linear code could be placed outside my expected enable interrupt and disable interrupt sequence (start/end points).

I may have misunderstood your question, but:

If you have code that looks like this:

Disable();
Do something;
Do something else;
Do a bit more;
Enable();

then you can be sure that the middle three lines are executed after the first line and before the last line.

 RE: How to make an interruptable code sequence in C?

Alas the code sequence did look exactly like this. Using a very aggressive optimization and looking at the list file I did see part of the code did get placed outside my disable enable sequence. Hence my question.

 RE: How to make an interruptable code sequence in C?

This is waaaaaaaay outside the scope of the 'C' Standard - therefore you are going to have to rely upon implementation-specific features.

 RE: How to make an interruptable code sequence in C?

"Alas the code sequence did look exactly like this."

I would be very interested to see the source code and the compiler's result.

I've never seen any compiler do anything different to what Jack illustrated.

 RE: How to make an interruptable code sequence in C?

"Alas the code sequence did look exactly like this."

I would be very interested to see the source code and the compiler's result.

Me too.

This is waaaaaaaay outside the scope of the 'C' Standard - therefore you are going to have to rely upon implementation-specific features.

Not necessarily. The standard mentions the so called sequence points. It is my understanding that those mandate a certain order of execution in a compiled program.
The general problem does exist, however. Interrupts and multiple CPU's do need more support in the language standard. I expect a new version of the standard in the future to include this.

 RE: How to make an interruptable code sequence in C?

The compiler wasn't Keil, but I was looking for opinions. Perhaps Keil considers the enabling/disabling interrupts as a sequence point.

The disable and enable functions were inlined automatically to be:

BCLR IEN ; global interrupt disable
BSET IEN ; global interrupt enable

The effect of using high optimization enabled the calls to be inlined thus eliminating the sequence points. Thus enabling/disabling interrupts in effect became simple reading/writing of memory locations from the compiler point of view with no dependency on the other variables (other than it was a volatile access). Enabling or disabling interrupts is not a FILE operation so no sequence point occurrence. The optimizer constraint was to make sure it was updated before hitting a sequence point. Unfortunately this resulted in undesirable behavior on my part. A volatile far pointer was being updated. But only part of the update was protected from interrupts and as luck would have it an interrupt occurred that used this pointer. A semaphore (mutex) was added in regards to this pointer to solve the issue (albeit we should have used one from the beginning).

Sorry, in my case I may have killed your Granny

 RE: How to make an interruptable code sequence in C?

The effect of using high optimization enabled the calls to be inlined thus eliminating the sequence points.

That would be a compiler bug, if I am not mistaken. It's not much consolation, though...

 RE: How to make an interruptable code sequence in C?

The compiler wasn't Keil, but I was looking for opinions.

Does the compiler by any chance have an optimization stage that works on the generated assembly code without regarding the C code? "Instruction reordering" or something similar?

 RE: How to make an interruptable code sequence in C?

A compiler normally has stages working on the instruction level. But these stages must know about side effects. And these stages should still have access to meta-data from the source-level processing.

But one problem with interrupts is that depending on where in the interrupt chain you perform your disable may affect if the disable is instant, or if there are one or more machine cycles after the disable instruction where you may still be hit by an interrupt.

When the processor has a machine instruction to turn off interrupts, it may have a pipeline where one or more following instructions may already have been affected by the previous interrupt enable state.

And when the disable is instead a write to an external interrupt controller, the internal busses in the processor may delay the write to the interrupt controller, allowing the processor to start new instructions.

Because of this, you really have to read the datasheets for the processor and may have to add one or more NOP instructions between the disable, and the critical code section. It is very hard to find information about the lag between disable and until you enter the safe zone. And it isn't easy to test either since it is almost impossible to trig an interrupt at the exact clock cycle that maximizes this delay in relation to the disable instruction.

 RE: How to make an interruptable code sequence in C?

Alas the code sequence did look exactly like this. Using a very aggressive optimization and looking at the list file I did see part of the code did get placed outside my disable enable sequence.

There are a number of optimisations such as 'common block subroutines' that will result in chunks of your translated code appearing in physically disparate places in the resulting assembly listing, this doesn't mean that those chunks are not executed after the disable() and before the enable(). It just means that the some of those instruction sequences also form part of the code elsewhere in your program. When they are executed as part of the code block you are concerned about they will be executed in the correct order.

Are you sure that you are seeing a real reordering or is it just the fact that bits of the code are all over the place that's giving that impression? Have you traced the flow through by hand or in the debugger to confirm?

 RE: How to make an interruptable code sequence in C?

To emphasize the point:

SwitchOffPower();
ConnectGrannyAcrossLiveAndNeutral();
DisconnectGranny();
SwitchOnPower();

I would be disappointed if my compiler reordered that one. Well, probably.

 RE: I would be disappointed if my compiler reordered that one

I guess it would depend on whether the compiler could determine any possible side-effects due to ConnectGrannyAcrossLiveAndNeutral() that might be affected by not having previously completed SwitchOffPower()...

=:0

 RE: I would be disappointed if my compiler reordered that one

The compiler can't reorder operations unless it has 100% control. Whenever it can't see the individual instructions of a called function, it has to assume that the function is un-touchable. If the function is inlineable, the compiler may look at individual instructions from the function and mix them - but still only when not changing order of secondary effects.

That is why the volatile keyword can be so important in multithreaded applications or when sharing variables with interrupt handlers. The compiler assumes that "normal" memory accesses can be reordered. A volatile variable that turns off the power will instantly tell the compiler that something magic happens that the compiler must not rearrange.

 RE: I would be disappointed if my compiler reordered that one

> That is why the volatile keyword can be so important in multithreaded
> applications or when sharing variables with interrupt handlers. The
> compiler assumes that "normal" memory accesses can be reordered. A
> volatile variable that turns off the power will instantly tell the
> compiler that something magic happens that the compiler must not
> rearrange.

volatile does not guarantee this for all situations. A nice
summary of everything that can go wrong with volatiles is
here: http://www.cs.utah.edu/~regehr/papers/emsoft08-preprint.pdf

Enjoy!

Marcus
http://www.doulos.com/arm/

 RE: I would be disappointed if my compiler reordered that one

Marcus,

A most outstanding article. Thanks!

 RE: I would be disappointed if my compiler reordered that one

An interesting article. However, I'm not quite sure that every description is correct. On page 2, right side, it states that the loop

for (i=0; i<BUF_SIZE; i++)
buffer[i] = 0;

does not have any side effects - but modifying _any_ object is defined as a side effect in my copy of the C standard (merely _accessing_ a volatile object is another side effect, calling a function that does either of the two is the third side effect). Hence the compiler would not be free to move the loop around.

A common mistake I could see would be something along the lines of

int foo, bar;
...
void some_function(void)
{
...
disable_interrupts();
foo = bar;
enable_interrupts();
...
}

If bar is not volatile, the compiler is free to move the read access (since read accesses to non-volatile variables don't count as side effects), and it may end up being read before the interrupts are disabled.

 RE: I would be disappointed if my compiler reordered that one

An interesting article. However, I'm not quite sure that every description is correct. On page 2, right side, it states that the loop

for (i=0; i<BUF_SIZE; i++)
buffer[i] = 0;

does not have any side effects - but modifying _any_ object is defined as a side effect in my copy of the C standard (merely _accessing_ a volatile object is another side effect, calling a function that does either of the two is the third side effect). Hence the compiler would not be free to move the loop around.

I agree with you, but find that I could argue the point either way based on what I read in different sections of the standard.

Someone ought to post this on comp.lang.c for a definitive answer.

 RE: I would be disappointed if my compiler reordered that one

Ok, this is a lengthy thread but well worth a read. The code in question is dealt with starting around the 50th post:

http://groups.google.com/group/comp.arch.embedded/browse_thread/thread/249c96ec8debf7a2/eec8b4a8060c510f

 What the hardware platform the OP is working on

=> Interrupts and multiple CPU's do need more support in the language standard. (Mike Kleshov) <=

I almost don't know anything about such an arduous subject. But I am still curious about what the hardware platform the OP is working on? Is it a Multi-core or Multithreading (hardware, like Hyper-threading) platform?

 RE: What the hardware platform the OP is working on

Is it a Multi-core or Multithreading (hardware, like Hyper-threading) platform?

It doesn't really have to be. A simple single core processor with interrupts is enough to cause the headaches mentioned by the OP.

 RE: What the hardware platform the OP is working on

Assuming that, there are some Parallel computing features exist on a specific platform.

Is it possible that?

part of the code get placed outside the disable enable sequence.

but,

Actually, those code are executed with the correct order during runtime,

because, the instructions are dispatched to different processing units.

 RE: What the hardware platform the OP is working on

Or maybe, it is an In-Order CPU (like Intel ATOM), so that the compiler has to provide some magic.

I just imagine, I don't have any real knowledge and experience on these arduous subjects. I even don't know how to code in assembly.

 RE: What the hardware platform the OP is working on

Processors with multiplie cores or intended to share memory has special instructions to force synchronization of cache contents, to make sure that writes aren't stuck in pipelines.

But that is a very different issue from the single-processor problem of synchronizing several software threads, or a main thread and an interrupt handler.

But in both situations, you often end up needing the use of assembler, to create code blocks that the compiler just can't fiddle with. And you need a "dumb" linker that doesn't try to reorder or rewrite code.

 RE: What the hardware platform the OP is working on

Actually, those code are executed with the correct order during runtime,

because, the instructions are dispatched to different processing units.

I don't think we're quite at a point where a CPU turn originally single-threaded code into something that's executed in several threads. That would be kind of a holy grail of parallel computing. ;)

 RE: What the hardware platform the OP is working on

A high-end x86 processor may spend a million transistors just for analyzing relationsships between instructions, and then reordering them and then splitting off the instructions into multiple ALU, FP, etc.

Some day, we may get similar behaviour in larger embedded processors too, but even when it can be done it would be impossible to keep track of tight timing so we would most probably still have "dumb" sequential processors for the hard real-time controller tasks, and use a superscalar, multi-core processor as a "back-end" computation engine responsible for crunching and non-critical tasks.

How do you prove something correct, when you have an infinite number of combinations that your source code can be converted into instructions, and these instructions can be sliced and diced between concurrently working execution units? A tiny little asynchronous interrupt (not to mention an exception) will completely rearrange the execution sequence.

Most probably, we will get better languages for describing concurrent and sequential operations, where the language will help with critical sections and concurrency. Even our embedded platforms have come a long way from the hardware platforms originally in existence when C was invented.

 Why the "volatile" type class should not be used

A Linux kernel documentation:

http://www.kernel.org/doc/Documentation/volatile-considered-harmful.txt

Why the "volatile" type class should not be used

Just for your reference. (I read the above kernel documentation, but I think I don't really understand it.)