1 <!-- Copyright (C) 2003 Red Hat, Inc. -->
2 <!-- This material may be distributed only subject to the terms -->
3 <!-- and conditions set forth in the Open Publication License, v1.0 -->
4 <!-- or later (the latest version is presently available at -->
5 <!-- http://www.opencontent.org/openpub/). -->
6 <!-- Distribution of the work or derivative of the work in any -->
7 <!-- standard (paper) book form is prohibited unless prior -->
8 <!-- permission is obtained from the copyright holder. -->
13 ><meta name="MSSmartTagsPreventParsing" content="TRUE">
16 CONTENT="Modular DocBook HTML Stylesheet Version 1.76b+
19 TITLE="eCos Reference Manual"
20 HREF="ecos-ref.html"><LINK
22 TITLE="The eCos Kernel"
23 HREF="kernel.html"><LINK
25 TITLE="Kernel Overview"
26 HREF="kernel-overview.html"><LINK
28 TITLE="Thread creation"
29 HREF="kernel-thread-create.html"></HEAD
40 SUMMARY="Header navigation table"
49 >eCos Reference Manual</TH
57 HREF="kernel-overview.html"
71 HREF="kernel-thread-create.html"
82 NAME="KERNEL-SMP">SMP Support</H1
90 >SMP -- Support Symmetric Multiprocessing Systems</DIV
94 NAME="KERNEL-SMP-DESCRIPTION"
99 >eCos contains support for limited Symmetric Multi-Processing (SMP).
100 This is only available on selected architectures and platforms.
101 The implementation has a number of restrictions on the kind of
102 hardware supported. These are described in <A
103 HREF="hal-smp-support.html"
104 >the Section called <I
110 >The following sections describe the changes that have been made to the
111 eCos kernel to support SMP operation.
117 NAME="KERNEL-SMP-STARTUP"
122 >The system startup sequence needs to be somewhat different on an SMP
123 system, although this is largely transparent to application code. The
124 main startup takes place on only one CPU, called the primary CPU. All
125 other CPUs, the secondary CPUs, are either placed in suspended state
126 at reset, or are captured by the HAL and put into a spin as they start
127 up. The primary CPU is responsible for copying the DATA segment and
128 zeroing the BSS (if required), calling HAL variant and platform
129 initialization routines and invoking constructors. It then calls
133 > to enter the application. The
134 application may then create extra threads and other objects.
137 >It is only when the application calls
140 >cyg_scheduler_start</TT
141 > that the secondary CPUs are
142 initialized. This routine scans the list of available secondary CPUs
145 >HAL_SMP_CPU_START</TT
147 CPU. Finally it calls an internal function
150 >Cyg_Scheduler::start_cpu</TT
151 > to enter the scheduler
155 >Each secondary CPU starts in the HAL, where it completes any per-CPU
156 initialization before calling into the kernel at
159 >cyg_kernel_cpu_startup</TT
160 >. Here it claims the
161 scheduler lock and calls
164 >Cyg_Scheduler::start_cpu</TT
170 >Cyg_Scheduler::start_cpu</TT
171 > is common to both the
172 primary and secondary CPUs. The first thing this code does is to
173 install an interrupt object for this CPU's inter-CPU interrupt. From
174 this point on the code is the same as for the single CPU case: an
175 initial thread is chosen and entered.
178 >From this point on the CPUs are all equal, eCos makes no further
179 distinction between the primary and secondary CPUs. However, the
180 hardware may still distinguish between them as far as interrupt
181 delivery is concerned.
187 NAME="KERNEL-SMP-SCHEDULING"
192 >To function correctly an operating system kernel must protect its
193 vital data structures, such as the run queues, from concurrent
194 access. In a single CPU system the only concurrent activities to worry
195 about are asynchronous interrupts. The kernel can easily guard its
196 data structures against these by disabling interrupts. However, in a
197 multi-CPU system, this is inadequate since it does not block access by
201 >The eCos kernel protects its vital data structures using the scheduler
202 lock. In single CPU systems this is a simple counter that is
203 atomically incremented to acquire the lock and decremented to release
204 it. If the lock is decremented to zero then the scheduler may be
205 invoked to choose a different thread to run. Because interrupts may
206 continue to be serviced while the scheduler lock is claimed, ISRs are
207 not allowed to access kernel data structures, or call kernel routines
208 that can. Instead all such operations are deferred to an associated
209 DSR routine that is run during the lock release operation, when the
210 data structures are in a consistent state.
213 >By choosing a kernel locking mechanism that does not rely on interrupt
214 manipulation to protect data structures, it is easier to convert eCos
215 to SMP than would otherwise be the case. The principal change needed to
216 make eCos SMP-safe is to convert the scheduler lock into a nestable
217 spin lock. This is done by adding a spinlock and a CPU id to the
221 >The algorithm for acquiring the scheduler lock is very simple. If the
222 scheduler lock's CPU id matches the current CPU then it can just increment
223 the counter and continue. If it does not match, the CPU must spin on
224 the spinlock, after which it may increment the counter and store its
225 own identity in the CPU id.
228 >To release the lock, the counter is decremented. If it goes to zero
229 the CPU id value must be set to NONE and the spinlock cleared.
232 >To protect these sequences against interrupts, they must be performed
233 with interrupts disabled. However, since these are very short code
234 sequences, they will not have an adverse effect on the interrupt
238 >Beyond converting the scheduler lock, further preparing the kernel for
239 SMP is a relatively minor matter. The main changes are to convert
240 various scalar housekeeping variables into arrays indexed by CPU
241 id. These include the current thread pointer, the need_reschedule
242 flag and the timeslice counter.
245 >At present only the Multi-Level Queue (MLQ) scheduler is capable of
246 supporting SMP configurations. The main change made to this scheduler
247 is to cope with having several threads in execution at the same
248 time. Running threads are marked with the CPU that they are executing on.
249 When scheduling a thread, the scheduler skips past any running threads
250 until it finds a thread that is pending. While not a constant-time
251 algorithm, as in the single CPU case, this is still deterministic,
252 since the worst case time is bounded by the number of CPUs in the
256 >A second change to the scheduler is in the code used to decide when
257 the scheduler should be called to choose a new thread. The scheduler
258 attempts to keep the <SPAN
265 > highest priority threads. Since an event or
266 interrupt on one CPU may require a reschedule on another CPU, there
267 must be a mechanism for deciding this. The algorithm currently
268 implemented is very simple. Given a thread that has just been awakened
269 (or had its priority changed), the scheduler scans the CPUs, starting
270 with the one it is currently running on, for a current thread that is
271 of lower priority than the new one. If one is found then a reschedule
272 interrupt is sent to that CPU and the scan continues, but now using
273 the current thread of the rescheduled CPU as the candidate thread. In
274 this way the new thread gets to run as quickly as possible, hopefully
275 on the current CPU, and the remaining CPUs will pick up the remaining
276 highest priority threads as a consequence of processing the reschedule
280 >The final change to the scheduler is in the handling of
281 timeslicing. Only one CPU receives timer interrupts, although all CPUs
282 must handle timeslicing. To make this work, the CPU that receives the
283 timer interrupt decrements the timeslice counter for all CPUs, not
284 just its own. If the counter for a CPU reaches zero, then it sends a
285 timeslice interrupt to that CPU. On receiving the interrupt the
286 destination CPU enters the scheduler and looks for another thread at
287 the same priority to run. This is somewhat more efficient than
288 distributing clock ticks to all CPUs, since the interrupt is only
289 needed when a timeslice occurs.
292 >All existing synchronization mechanisms work as before in an SMP
293 system. Additional synchronization mechanisms have been added to
294 provide explicit synchronization for SMP, in the form of
296 HREF="kernel-spinlocks.html"
304 NAME="KERNEL-SMP-INTERRUPTS"
307 >SMP Interrupt Handling</H2
309 >The main area where the SMP nature of a system requires special
310 attention is in device drivers and especially interrupt handling. It
311 is quite possible for the ISR, DSR and thread components of a device
312 driver to execute on different CPUs. For this reason it is much more
313 important that SMP-capable device drivers use the interrupt-related
314 functions correctly. Typically a device driver would use the driver
315 API rather than call the kernel directly, but it is unlikely that
316 anybody would attempt to use a multiprocessor system without the
320 >Two new functions have been added to the Kernel API
322 HREF="kernel-interrupts.html#KERNEL-INTERRUPTS-SMP"
327 >cyg_interrupt_set_cpu</TT
331 >cyg_interrupt_get_cpu</TT
332 >. Although not currently
333 supported, special values for the cpu argument may be used in future
334 to indicate that the interrupt is being routed dynamically or is
335 CPU-local. Once a vector has been routed to a new CPU, all other
336 interrupt masking and configuration operations are relative to that
340 >There are more details of how interrupts should be handled in SMP
342 HREF="devapi-smp-support.html"
343 >the Section called <I
354 SUMMARY="Footer navigation table"
365 HREF="kernel-overview.html"
383 HREF="kernel-thread-create.html"
projects / karo-tx-redboot.git / blob