CORBA Control Systems - UPM ASLab

Transcripción

T
HE
A
SYSTEMS LABORATORY
www.aslab.org
UTONOMOUS
CORBA Control Systems
A Distributed Object Approach to Complex Control
Engineering
Miguel José Segarra
R-2005-012
2005-7-19
UNIVERSIDAD POLITÉCNICA DE MADRID
ESCUELA TÉCNICA SUPERIOR DE INGENIEROS INDUSTRIALES
CORBA CONTROL SYSTEMS
PhD Thesis
Miguel José Segarra Martínez
Ingeniero Industrial
2005
This page has been intentionally left blank
UNIVERSIDAD POLITÉCNICA DE MADRID
ESCUELA TÉCNICA SUPERIOR DE INGENIEROS INDUSTRIALES
Departamento de Automática, Ingeniería Electrónica e Informática Industrial
PhD Thesis
Author:
Director:
MIGUEL JOSÉ SEGARRA MARTÍNEZ
RICARDO SANZ BRAVO
Doctor Ingeniero Industrial
Madrid, 2005
Título:
Autor:
MIGUEL JOSÉ SEGARRA MARTÍNEZ
TRIBUNAL
Tribunal nombrado por el Mgfco. y Excmo. Sr. Rector de la Universidad Politécnica
de Madrid el día 22 de Junio de 2005.
Presidente:
Dr. Ramón Galán López
Vocales:
Dr. Santos Galán Casado
Dra. Miren Idoia Alarcón Rodríguez
Dr. Ángel de Antonio Bermejo
Secretario:
Dr. Manuel Rodríguez Hernández
Suplentes:
Dr. Juan Ramón Velasco Pérez
Dra. Marga Marcos Muñoz
Realizado el acto de la lectura y defensa de la Tesis el día 19 de Julio de 2005.
Calificación de la Tesis......................
El Presidente
El Secretario
Los Vocales
For Cloé and Hugo.
Para Cloé y Hugo.
Pour Cloé et Hugo.
― Miguel
Acknowledgments
This has always looked to me one of the hardest parts to write of any book. It always
seems there is not a fine way to thank all the people who have worked with me in
many different areas over the last years. For sure, some of you will not find your name
here. It does mean neither that you have not contributed to what I know nor that you
have not contributed to this Thesis in one way or another. Simply, I cannot recall all
the important things many people have taught me and that have been used in my work.
Consider yourself acknowledged in these lines if your name just slipped from my
mind. Of course, I am acknowledging the people I can remember.
I will start with my co-workers. José Antonio Clavijo, my partner at SCILabs
Ingenieros, has spent the last few years with me trying to build up the business. It is
tough thing to do and we have spent many days working around the clock (literally) in
order to hit the deadline. We never had a serious complaint from a customer and we
are proud our software is involved in the construction of some of the largest
infrastructures of Spain. We have built together the new version of the ICa ORB. José
Antonio, your turn is next for showing up your Thesis work.
Carlos Moreno is another co-worker at SCILabs that recently left to work in the
Telecom industry. In spite of that, we are friends and he has been with José Antonio
and me from the very beginning. He has also worked very hard with us and knows
well the late hours of night at the office. I would have liked he could remain with us at
SCILabs. Eventually, he became very pushy with me about finishing this Thesis work
which I appreciate.
Angel de Antonio, Associate Professor of the Universidad Autónoma de Madrid,
developed the first version of ICa. That first ORB, without being fully CORBA
compliant but following its spirit, was a large leap from the previous communications
software being employed at ASLAB. I spent with him many hours and days testing
and debugging many pieces of that initial software. That ORB was the precursor of
what the ICa ORB is today. Probably, the ICa ORB would not be what it currently is
without that initial work. I have to thank him for the good job.
The European Union IST programme has a lot to do with the funding of this work. I
possibly could not finish this Thesis without its support. I think the help received from
the EU has not only been economic. Behind funding and bureaucracy there are people
that might believe in what you are doing or that might see an opportunity for the
technical, social and economical development of Europe. Consequently, I would like
to thank these people I have found abroad and that have certainly contributed to
CORBA Control Systems.
First, from the European Commission, I want to thank Alkis Konstantellos who is part
of the Embedded Systems Unit of IST. When SCILabs spun off I already had some
background in the CORBA world from my previous work at ASLab-UPM (mainly
from the DIXIT and Hydra projects). By that time, a Spanish PLC manufacturer and
engineering firm, ELIOP, contacted SCILabs to participate in an IST project where
CORBA technology was involved. The project was DOTS (more on DOTS can be
found in the Thesis). This project, which SCILabs’ part I directed, was the first
international consortium of the firm. I remember the first review meeting in Rome
(Italy) was highly stressing for me. There were lots of things that could go wrong as
we had just started the company and had neither a good financial support nor business
background experience to prove our capabilities. However, it turned out that Dr.
Konstantellos, at that moment the project officer from the EU, preferred to make
decisions based on facts and I had my chance to prove what SCILabs could do. The
real-time ICa ORB was developed during the DOTS project. Alkis Konstantellos
trusted SCILabs and it did not deceive him. He also deserves acknowledgement in
these lines.
I would also like to thank Professor Hermann Härtig, one of the DOTS project
reviewers. At the beginning of the project, he was not sure that SCILabs could meet
the deadline. He openly said so in the DOTS first review meeting held in Rome (Italy)
which I refuted by exposing that all the tasks were carefully planned and scheduled.
By the end of the project, he was greatly surprised by the amount of work achieved.
Later, after the end of DOTS, he invited me as a speaker to the SDA 2002 (System
Design Automation Workshop) at the Technical University of Dresden (Germany),
where I spent a wonderful time with the people at the Computer Science department.
He was very helpful in all reviews and tried to supply positive ideas and
recommendations for the ongoing work. Also, after the project, he offered me to
integrate the ICa ORB with the Fiasco Microkernel being developed at Dresden
Technical University. At the time of the offer, I could not spare the needed resources
to do it but I do not forget his offering and maybe it is possible in the future.
I want to show my appreciation for the work carried out in DOTS by some of ELIOP’s
technical staff. The hardware for the new RTUs of ELIOP was also developed during
the project which put additional stress (and risk) into the schedule as the machine and
the broker were new and, both had to be integrated with the application software.
Ignacio González and Fabrice Wawak did a great work to solve many problems
related to hardware and basic software of the RTU. I hope both of you have already
forgotten that little code bug in the Ethernet chip of the CNI. Also, Francisco
Rodríguez did a great job as overall project coordinator of DOTS by constantly
remembering everybody their short term duties.
The work done in the DOTS project grew the confidence of the European Commission
in the possibilities of ICa for the distributed control purpose and, an ambitious project
was prepared, Hard Real-Time CORBA, in which SCILabs, UPM and two worldleading universities in Computing and Control theory, the University of Lund
(Sweden) and the Technical University of Vienna (Austria) participated. From the
University of Lund, I want to acknowledge Professor Åström. I had the opportunity to
show him the work being done in our team at ASLAB when he came to Madrid for the
homage to Professor Puente. He said that our team at ASLAB was “very aggressive
with work” which is very nice to hear from someone so respected. From this visit,
Professor Sanz kept in contact with the people of Lund University and thus, after
several years and interim activities with them, we were able to launch a joint project.
Also I would like to mention Professor Årzén and Associate Professor Nilsson from
Lund University, management colleagues of the HRTC project, for their valuable
contributions and insights regarding networked control systems.
From the Technical University of Vienna, I want to recognise the contribution of
Professor Kopetz, the man behind TTP. Once he is convinced of his ideas, he defends
them in a very sternly and stubborn way. Having him in the project always stirred the
pot and there were interesting discussions in project meetings about linking interfaces,
temporal firewalls and many other things. Also Thomas Losert, a PhD student of the
Technical University of Vienna, helped in the understanding of TTP and on the
possible mapping to a CORBA transport plug-in.
The HRTC project also counted with two good reviewers. First, Professor Thomson
from the University of Sheffield made interesting contributions and comments on the
work of HRTC. Professor Thomson has a closed English accent leading to some funny
situations during the project for non-UK citizens. I will make a proposal in my next
international project for non-UK people to take a British English course so we can
speak and understand real English. Secondly, Dr. Virginie Watine from Thales
Communications who has a deep understanding of CORBA and is very involved with
OMG. She clearly understood that the ICa ORB is the only ORB in Europe (probably
in the world) dealing with the problem of hard real-time. I also thank her for inviting
me, after HRTC concluded, to Thales in Paris (France) to the initial meetings for the
preparation of a project on CORBA components.
From UPM I would like to thank Professor Ricardo Sanz. It is many years now that
we know each other and we have developed a bond of friendship. He has helped me
enormously either academically or in my professional career. Certainly, it would be
impossible to provide here a list of the many things he has taught me or in which he
provided direction to me. Also, I thank him for counting on SCILabs for the
development of international research projects and I am happy we have reached a
framework of agreement for the research use of ICa by his team at ASLab. Of many
things, I would like to point out the extreme availability of Ricardo to whatever the
request I made him. Really, he puts up with a lot of different things at the same time
and, as you know, it is difficult to maintain a high level of context switching and be
productive. I can certify he is. Just one tip of advice for him, Ricardo, look at your
travel agenda, you cannot get into more things!
I would like also to give thanks to my family. They always support me whatever the
decision I make. My deepest gratitude to my parents, Miguel and Francisca, for giving
me the opportunity of having a good education. Also thanks to my sisters, Carmen and
Sonia, and to Cloé and Hugo, Sonia’s little daughter and son that make life joyful
when they come from Paris.
Last, but not least, my girlfriend (I am not married and cannot find a better word)
Rosanna pushed me to go on when my spirit ran short and morale went low. I thank
her for grounding me in reality, for her endurance and persistence in making me finish
this work and, most of all, for being my friend.
Mayo de 2005
Agradecimientos
Esta siempre me ha parecido una de las partes más difíciles de escribir de cualquier
libro. Siempre parece que no existe una buena manera de dar las gracias a toda la
gente que ha trabajado conmigo en muchas áreas diferentes durante los últimos años.
A buen seguro, algunas de esas personas no encontrarán su nombre en estas líneas.
Eso no quiere decir que no hayan contribuido a lo que sé o que no hayan contribuido a
la realización de esta Tesis de una manera u otra. Sencillamente, no me resulta
posible recordar todas las cosas importantes que me habéis enseñado y que se han
utilizado en mi trabajo. Considerad vuestra aportación reconocida en estos
agradecimientos si olvido mencionaros. Por supuesto, voy a dar las gracias a las
personas que sí recuerdo.
Empezaré con mis compañeros de trabajo. José Antonio Clavijo, mi socio en SCILabs
Ingenieros, ha pasado los últimos años conmigo tratando de sacar adelante la empresa.
Es algo verdaderamente difícil de hacer y hemos pasado muchos días enteros
trabajando (literalmente) para cumplir con los compromisos que habíamos adquirido.
Nunca hemos tenido una queja seria de ningún cliente y estamos orgullosos de que
nuestro software sirva para la construcción de algunas de las infraestructuras más
grandes de España. Hemos construido juntos la nueva versión del ORB ICa. José
Antonio, tu turno es el próximo para presentar tu trabajo de Tesis Doctoral.
Carlos Moreno es otro trabajador de SCILabs que recientemente nos dejó para trabajar
en la industria de las Telecomunicaciones. A pesar de esto, somos amigos y ha estado
con José Antonio y conmigo desde el principio. También ha trabajado muy duro y
conoce perfectamente las horas tardías de la noche en la oficina. Me hubiese gustado
que siguieras con nosotros en SCILabs. Desde hace unos meses ha sido muy insistente
para que terminara esta Tesis, cosa que le agradezco.
Ángel de Antonio, Profesor Asociado de la Universidad Autónoma de Madrid,
desarrolló la primera versión del broker ICa. Ese primer ORB, que aunque no cumplía
completamente con la especificación de CORBA seguía su espíritu, fue un gran salto
adelante en comparación con el software de comunicaciones que usábamos
previamente en ASLab. Empleé muchas horas y días con él, probando y depurando
muchos componentes de aquel software inicial. Aquel ORB fue el precursor de lo que
ICa ORB es hoy. Probablemente, la versión actual de ICa ORB no existiría sin aquel
desarrollo inicial. Le agradezco que realizara un buen trabajo.
El programa IST de la Unión Europea tiene mucho que ver con la financiación de este
trabajo. Probablemente, no hubiera podido acabar esta Tesis sin su apoyo. Creo que el
apoyo de la Unión Europea no ha sido sólo económico. Detrás de la financiación y la
burocracia hay personas que pueden creer en lo que uno hace o que pueden ver una
oportunidad para el desarrollo técnico, social o económico de Europa.
Consecuentemente, me gustaría dar las gracias a esas personas que he encontrado
fuera de mi país y que, ciertamente, han contribuido a los Sistemas de Control
CORBA.
Primero, perteneciente a la Comisión Europea, me gustaría agradecer su apoyo a Alkis
Konstantellos, quien forma parte de la Unidad de Sistemas Empotrados del programa
IST. Cuando SCILabs surgió como empresa, yo ya tenía experiencia en el mundo de
CORBA a través de mi trabajo previo en ASLab-UPM (fundamentalmente de los
proyectos DIXIT e Hydra). Por esa época, un fabricante español de PLCs se puso en
contacto con SCILabs para ofrecernos participar en un proyecto IST donde la
tecnología CORBA era necesaria. El proyecto era DOTS (más sobre DOTS se puede
encontrar en la Tesis). Este proyecto, en el que dirigí la participación de SCILabs, fue
el primer consorcio internacional de la empresa. Recuerdo que la primera reunión de
revisión del proyecto en Roma (Italia) fue muy estresante para mí. Había muchas
cosas que podían ir mal ya que prácticamente habíamos comenzado nuestra andadura
como compañía y no contábamos ni con un buen soporte financiero ni con años de
experiencia empresarial que permitieran demostrar nuestra capacidad. Sin embargo,
ocurrió que el Dr. Konstantellos, en aquel momento el oficial de la Comisión Europea
a cargo del proyecto, prefirió tomar decisiones basándose en hechos y tuve la
oportunidad de demostrar lo que SCILabs podía hacer. El ORB de tiempo real de ICa
fue desarrollado a lo largo del proyecto DOTS. Alkis Konstantellos confió en SCILabs
y ésta no le decepcionó. El también merece mi agradecimiento en estas líneas.
También quiero dar las gracias al Profesor Hermann Härtig, uno de los revisores
nombrados por la Comisión Europea para proyecto DOTS. Al comienzo de DOTS, él
no estaba convencido de que SCILabs pudiera cumplir los plazos del proyecto. Lo
comentó abiertamente en la primera revisión del proyecto en Roma (Italia) y yo le
refuté exponiendo que todas las tareas estaban cuidadosamente organizadas y
planificadas. Al final del proyecto, estaba absolutamente sorprendido de todo el
trabajo realizado. Más tarde, tras la finalización del proyecto DOTS, me invitó a dar
una conferencia en el SDA 2002 (System Design Automation workshop) en la
Universidad Técnica de Dresde (Alemania), donde pasé unos días realmente
agradables con los profesores y estudiantes de postgrado del Departamento de
Ciencias de la Computación. Hermann Härtig supo ser de utilidad en todas las
revisiones del proyecto DOTS, tratando siempre de proporcionar ideas y
recomendaciones positivas para el trabajo en desarrollo. También, después del
proyecto DOTS, me ofreció la posibilidad de integrar el ORB ICa con el microkernel
Fiasco que se estaba desarrollando en la Universidad Técnica de Dresde. En aquel
momento tuve que desechar la oferta ya que no disponía de los recursos necesarios
para hacerlo. Quizás sea posible en el futuro.
También quiero mostrar mi aprecio al personal técnico de ELIOP por el trabajo
realizado en DOTS. El hardware de las nuevas RTUs de ELIOP se desarrolló también
durante el proyecto, lo que introdujo presión y riesgo adicional a su planificación ya
que la máquina utilizada e ICa ORB eran nuevos, y ambos debían integrarse con el
software de aplicación. Ignacio González y Fabrice Wawak hicieron un gran trabajo
para resolver muchos problemas relacionados con el hardware y el software básico de
las RTU. Espero que ya os hayáis olvidado de aquel pequeño bug del código del chip
Ethernet del CNI. Agradezco también a Francisco Rodríguez su gran trabajo como
coordinador general de DOTS, recordando insistentemente a todos la cercanía del
cumplimiento de nuestras obligaciones más inmediatas.
El trabajo realizado en el proyecto DOTS hizo que creciera la confianza que la
Comisión Europea depositó en las posibilidades de ICa para su uso en sistemas de
control distribuido y, en consecuencia, se preparó un proyecto ambicioso, Hard RealTime CORBA, en el que SCILabs, la UPM y dos Universidades líderes mundiales en
ciencias de la computación y teoría de control, la Universidad de Lund (Suecia) y la
Universidad Técnica de Viena (Austria) participaron. De la Universidad de Lund
quiero mostrar mi agradecimiento al Profesor Åström. Tuve la oportunidad de
mostrarle el trabajo que llevábamos a cabo en nuestro grupo de trabajo en ASLab
cuando vino a Madrid para participar en el homenaje al Profesor Puente. Comentó que
nuestro grupo “era muy agresivo con el trabajo” lo que sin duda es muy agradable
escuchar cuando viene de alguien tan respetado. A partir de esta visita, el Profesor
Sanz mantuvo el contacto con los equipos de investigación de Lund y de esta manera,
tras varios años y diversas actividades intermedias con ellos, surgió la posibilidad de
lanzar un proyecto conjunto. También me gustaría mencionar al Profesor Årzén y al
Profesor Nilsson, colegas de gestión del proyecto HRTC, por sus valiosas
contribuciones y puntos de vista sobre los sistemas de control en red.
De la Universidad Técnica de Viena, quiero agradecer la contribución del Profesor
Kopetz, el hombre tras el protocolo TTP y la arquitectura TTA. Una vez está
convencido de sus ideas las defiende de una manera tenaz, incluso obstinada. Su
participación en el proyecto HRTC siempre fue un estímulo y hubo discusiones
interesantes en las reuniones del proyecto sobre interfases de enlace, cortafuegos
temporales y muchas otras cosas. También Thomas Losert, un estudiante de postgrado
de la Universidad Técnica de Viena, fue de ayuda en la comprensión del protocolo
TTP y en su posible implementación como un plug-in de transporte para ICa ORB.
El proyecto HRTC también contó con dos buenos revisores de la Unión Europea.
Primero, el Profesor Thomson de la Universidad de Sheffield realizó contribuciones y
comentarios interesantes sobre el trabajo realizado en HRTC. El Profesor Thomson
tiene un cerrado acento Inglés, lo que produjo algunas situaciones divertidas durante el
proyecto HRTC para los no Británicos. En mi próximo proyecto internacional haré
una propuesta para que los no pertenecientes al Reino Unido tomemos un curso de
Inglés Británico. De esta manera podremos hablar y entender el auténtico idioma
Inglés. En segundo lugar, la Dra. Virginie Watine de Thales Communications que
posee un profundo conocimiento de CORBA y está muy involucrada en las tareas del
OMG. Claramente comprendió que ICa es el único ORB en Europa (y probablemente
en el mundo) que ataca el problema del tiempo real crítico. Le agradezco que me
invitara, al concluir el proyecto HRTC, a Thales en París (Francia) para participar en
las reuniones preparatorias de un proyecto sobre componentes CORBA.
De la Universidad Politécnica de Madrid quiero dar las gracias al Profesor Titular
Ricardo Sanz. Son muchos años ya los que nos conocemos y hemos desarrollado un
lazo de amistad. Me ha ayudado y aconsejado enormemente tanto académicamente
como en mi carrera profesional. Sería imposible proporcionar aquí una lista de las
muchas cosas que me ha enseñado o a las que ha proporcionado dirección. También le
agradezco que haya contado con la participación de SCILabs para el desarrollo de
proyectos internacionales de investigación y me alegra que hayamos llegado a un
acuerdo para el uso de ICaORB por su equipo UPM-ASLab para proyectos de
investigación. Entre muchas cosas, me gustaría destacar la extrema disponibilidad de
Ricardo ante cualquier petición que le realizara. En realidad, él lleva una multitud de
asuntos de manera simultánea y, como sabéis, es difícil mantener un grado elevado de
cambios de contexto y ser productivo al mismo tiempo. Puedo decir que él lo es. Sólo
un pequeño consejo para él. Ricardo, mira tus viajes en la agenda - ¡No puedes
meterte en ninguna cosa más!
También quiero dar las gracias a mi familia. Siempre me apoyan tome la decisión que
tome. Mi más profunda gratitud a mis padres, Miguel y Francisca, por darme la
oportunidad de disfrutar de una buena educación. También doy las gracias a mis
hermanas, Carmen y Sonia, y a Cloé y Hugo, los pequeños hija e hijo de mi hermana
Sonia que hacen la vida más gozosa cada vez que vienen de Paris.
Por último, y desde luego no el menor, quiero dar mi agradecimiento a mi novia (no
estoy casado y no encuentro una palabra mejor), Rosanna, que me ha empujado a
seguir cuando fallaba el espíritu y la moral estaba baja. Le agradezco el ponerme los
pies en el suelo, su resistencia, persistencia y paciencia para conseguir que acabe este
trabajo y, sobre todo, que sea mi amiga.
Mayo de 2005
Resumen
Hoy en día la mayor parte de los sistemas de control están compuestos por otros
sistemas que forman un todo heterogéneo en un entorno distribuido. Son sistemas
complejos en los que partes de cada uno de los sistemas-componente se combina con
el resto para proporcionar funciones globales. La heterogeneidad surge porque existen
equipos especializados que realizan mejor ciertas tareas del sistema: lectura de
sensores o control de actuadores, gestión del proceso o gestión empresarial se sitúan
en diferentes niveles del sistema de control distribuido y realizan diferentes tareas.
En el caso de los sistemas de control, la razón principal para la distribución del
sistema es que el proceso de cálculo del algoritmo de control se sitúe cerca del
proceso. De esta manera, se asegura una reacción temprana ante los cambios del
mismo. Un ejemplo de este tipo de grandes sistemas de control distribuido son las
plantas de proceso, cuyo máximo exponente pueda ser quizás la industria
petroquímica. También existen otras razones que hacen que los sistemas de control
sean distribuidos.
•
•
•
•
•
•
•
•
Disponibilidad de procesadores para sistemas empotrados.
Requisitos temporales que hacen prohibitiva la utilización de comunicaciones
debido a las latencias.
La necesidad de proporcionar un elevado rendimiento del sistema por lo que
se recurre a la utilización de paralelismo en la computación.
La simplificación de las tareas de construcción y mantenimiento mediante la
utilización del concepto de modularidad.
La reducción del coste y tiempo de puesta en el mercado mediante la
reutilización de componentes.
La integración con sistemas propietarios existentes en la planta.
La disponibilidad de plataformas software específicas para algunos
propósitos.
Fiabilidad y tolerancia a fallos.
El ejemplo de la planta de proceso se puede trasladar fácilmente a otros dominios en
los que la heterogeneidad y distribución de los sistemas aparecen de manera natural.
El lector puede pensar en los modernos sistemas de combate donde la información
fluye desde muchas fuentes diferentes (radar, satélites, sistemas de adquisición de
blancos, etc.) y es recogida por una cantidad similar de sistemas sumidero (tropas,
aviónica, sistemas de posicionamiento, etc.). Otro ejemplo se encuentra en la industria
del automóvil. Los modernos sistemas de ABS tienen un computador dedicado al
control de cada rueda que intercambia la información de frenado y deslizamiento con
el resto. Además, la suspensión y el sistema de gestión del motor colaboran con el
ABS para mantener el automóvil bajo control. Aunque estos sistemas son comunes en
la actualidad, presentan la característica de que son difíciles de construir.
Mientras que el coste del hardware decrece a lo largo del tiempo, el software que se
ejecuta sobre él se hace cada vez más complejo y el esfuerzo y capital necesarios para
construirlo continúa aumentando. La heterogeneidad y distribución de los sistemas
contribuyen en gran medida al incremento de los costes y a la dificultad asociados a la
construcción del software. Por ejemplo, la construcción del software supone el 50%
del coste del nuevo Airbus 380.
En esta Tesis se ha realizado una investigación sobre la tecnología CORBA (Common
Object Request Broker Architecture) para atacar los problemas descritos en el campo
de los sistemas distribuidos de control. Es un campo amplio. Los sistemas de control
varían desde los muy pequeños, empotrados y de recursos limitados, hasta los muy
grandes, que abarcan una o varias plantas de proceso y contemplan desde el control
del proceso en sí mismo hasta la gestión de los pedidos a los proveedores. Sus
requisitos también varían del mismo modo, desde los temporalmente críticos hasta
aquellos en los que el tiempo en el que ocurren las acciones de control no es el
condicionante primordial. En una situación ideal, la mejor manera de solventar los
problemas de estos sistemas es sencillamente hacerlos desaparecer en la etapa de
construcción del sistema de control. Para conseguir ese propósito se necesita una
abstracción común a una amplia gama de sistemas de control. La abstracción común
debe ser un mecanismo independiente de las plataformas de manera que la
heterogeneidad pueda ser eliminada del sistema. Además, para prescindir de la
complejidad impuesta por la distribución, el sistema debería ser desarrollado como si
fuera un sistema monolítico. Estas son las ideas que subyacen tras la OMA (Object
Management Architecture). La Arquitectura de Gestión de Objetos proporciona una
abstracción sobre los problemas de bajo nivel inherentes a cada plataforma y
proporciona transparencia a los sistemas en cuanto a la distribución y heterogeneidad
de una manera orientada a objetos. La concreción de esa abstracción se consigue
mediante un modelo de objetos basados en la OMA. Un ejemplo de esta concreción es
la especificación CORBA. Las implementaciones de la especificación CORBA
proporcionan a los desarrolladores de sistemas una infraestructura homogénea e
independiente de la plataforma para la realización de su trabajo.
Mientras que la abstracción de CORBA ha probado su utilidad en algunos dominios
de aplicación, el modelo no es lo suficientemente bueno en algunas situaciones.
CORBA se diseñó con los objetivos de flexibilidad, interoperabilidad y reducción de
la complejidad sin incurrir en una pérdida excesiva de rendimiento.
Desafortunadamente, existen requisitos en algunas aplicaciones, como los sistemas de
control, respecto a las características de tiempo real, tolerancia a fallos o capacidad
para despliegue en sistemas embarcados o empotrados que no pueden dejarse de lado
si existe la pretensión de que el sistema opere en el mundo real. Se hace necesario
pues un análisis y una implementación de la infraestructura necesaria para el
desarrollo de este tipo de sistemas. Como base fundamental para este trabajo, y dada la
tecnología elegida como infraestructura (CORBA), se utiliza la Arquitectura Software
como elemento de referencia a la hora de modelar sistemas que sean capaces de
cumplir con los requerimientos que de ellos se exigirán. La tecnología de objetos y
objetos distribuidos se utiliza como medio para realizar representaciones abstractas del
mundo y como infraestructura de comunicación entre los mismos.
La Arquitectura del Software es importante porque aparece en las primeras etapas del
ciclo de vida del sistema y ayuda a tomar decisiones sobre su composición y
estructura, y sobre las relaciones existentes entre los elementos del mismo. La
definición de una arquitectura software ayuda a establecer compromisos entre las
capacidades técnicas que disfrutará un sistema determinado. De aquí la importancia
del uso de arquitecturas sólidas para el diseño de sistemas de control basados en
CORBA.
La tecnología de objetos es importante porque permite expresar de una manera
sencilla la arquitectura software de los sistemas. Los objetos encapsulan su estado y
proporcionan una interfaz para acceder al mismo. Los objetos también ayudan a
capturar las entidades esenciales de cualquier sistema y sus propiedades, permitiendo
dividir el sistema en partes que son manejables por las personas. Cuando a la
tecnología de objetos le añadimos la capacidad de que éstos se encuentren distribuidos
en diferentes piezas hardware o software mediante una infraestructura que les permite
colaborar de una manera natural en el sistema distribuido, disponemos de una
herramienta que permite construir sistemas complejos, heterogéneos y distribuidos de
una manera sencilla y rápida.
Con estas ideas en mente se desarrolla esta Tesis doctoral en la que se persiguen dos
objetivos fundamentales.
•
•
La investigación de la aplicación de la tecnología CORBA para la resolución
de los problemas de control asociados a la heterogeneidad, distribución,
complejidad, escalabilidad y falta de rendimiento de los sistemas actuales
basados en CORBA. Este trabajo se ha realizado dentro del marco de los
grandes sistemas industriales así como en sistemas empotrados de recursos
reducidos. Se han construido algunas extensiones a las especificaciones de
CORBA para tiempo real y se ha co-desarrollado un broker con el propósito
exclusivo del control industrial. Como validación de los trabajos realizados
existen numerosos ejemplos de aplicación desarrollados en los últimos años.
El segundo objetivo es el desarrollo de una metodología para la construcción
de sistemas de control distribuido basados en ICa ORB, el broker CORBA de
tiempo real en el que se basan los trabajos aquí expuestos. El propósito de la
metodología es la reducción del tiempo y esfuerzo dedicados a la
construcción de este tipo de sistemas.
La Tesis se encuentra organizada en ocho capítulos.
El primer capítulo sirve de introducción. Contiene información sobre las propiedades
que exhiben los sistemas de control distribuido. Como se ha comentado en las páginas
anteriores la heterogeneidad, complejidad y la necesidad de integración se encuentran
entre los factores más importantes a considerar.
El segundo capítulo, Sistemas de Control Industrial y Protocolos de Comunicación,
realiza un repaso del estado del arte de los sistemas distribuidos de control que se
encuentran disponibles de manera comercial. También se realiza un repaso de los
protocolos de comunicación más utilizados en la industria de control. Como se ha
indicado, el principal problema de la construcción de este tipo de sistemas es la
complejidad. La dificultad de dar solución al problema de control en sí mismo se ve
aumentada por el hecho de construir un sistema distribuido e intensivo en software.
El segundo gran problema es el de la integración. El sistema de control distribuido no
puede ser cerrado, multitud de sistemas diferentes deben coexistir. La información
debe fluir a través de todos los subsistemas de la planta sea cual sea su objetivo:
operacional, táctico o estratégico. La integración siempre resulta difícil y costosa e
incluso son los propios fabricantes los que ponen barreras tecnológicas con el fin de
promocionar sus propios productos. La Arquitectura de Control Integrado (ICa)
soluciona este problema proporcionando un marco de integración común a todos los
sistemas que intervienen en el control.
El tercer gran problema es el de la flexibilidad. Muchos sistemas de control se
construyen a partir de un diseño ad-hoc, específico para el problema a solucionar en
un momento dado del tiempo y que imponen una estructura rígida a la forma en que se
opera el proceso. También es necesario considerar el futuro y las cambiantes
condiciones del mercado, donde será necesario adaptar el entorno tan pronto como sea
posible a nuevas necesidades de producción y siempre antes que la competencia. La
capacidad de adaptación del sistema incurrirá en mayor o menor medida a los costes
de evolución de la planta. Tecnologías como ICa permiten reducir el esfuerzo
necesario para la adaptación de los sistemas que deben evolucionar o cambiar.
El cuarto gran problema es el de la estandarización. Por ejemplo, los fabricantes de
sistemas de automatización hacen grandes esfuerzos para distinguir las interfases de
sus productos y tratan de imponer sus “estándares” de tecnología lo que conlleva la
dependencia de un solo fabricante. ICa soluciona este problema mediante una interfase
de interoperabilidad común acordada por más de 500 empresas de todo el mundo.
Dentro del segundo capítulo también se describen los tipos de sistemas de control en
los que la utilización de ICa puede aportar beneficios. En un principio, la idea
fundamental consistía en resolver el problema de la integración en grandes sistemas de
control. Tras las primeras versiones del gestor de objetos se observó la posibilidad de
extender ICa a sistemas de control distribuido con otras características.
Fundamentalmente, sistemas distribuidos y empotrados con características de tiempo
real ya fueran críticas o no. De esta manera, existen diferentes versiones del ICa ORB
que pueden embeberse bien en grandes sistemas o en sistemas empotrados con
características de tiempo real muy diferentes, manteniendo la gran ventaja de la
interoperabilidad entre todas ellas. Esto permite obtener una verdadera integración
total, tanto vertical como horizontal en todas las capas de la jerarquía de control. En
ICa se emplea el término Total Integration para referirse a esta estrategia.
La estructura organizativa que más ha penetrado en el diseño de los sistemas de
control es la arquitectura en capas. En el caso del control distribuido es una
consecuencia natural de los diferentes tipos de tareas que se deben realizar en el
sistema. La estructura en capas permite organizar lógicamente desde los lazos básicos
de control hasta planificación de la producción de la planta desde el punto de vista
administrativo o de la gestión de los proveedores. Las capas del sistema de control son
un medio para reducir la complejidad del mismo. Las funciones similares se agrupan
en las mismas capas y se desarrollan independientemente del resto. La comunicación
se limita a las capas adyacentes con lo que se reduce el acoplamiento entre las partes
del sistema, aumentando al mismo tiempo la reusabilidad del software. Las capas
ofrecen una interfaz de servicios que es utilizada por las capas superiores para realizar
las tareas de control. A su vez, en los sistemas de control, las capas inferiores
proporcionan información a las superiores sobre el estado del proceso. Este
mecanismo de comunicación puede ser implementado, por ejemplo, en la forma de un
gestor de eventos.
En general, la estructura en capas corresponde a un punto de vista del sistema de
control en el que simplemente se ha realizado una división lógica de la funcionalidad
del mismo. Existe otro punto de vista arquitectónico del sistema en el que el énfasis se
encuentra en la implementación de cada una de las capas que componen el sistema de
control. Así que en realidad, la arquitectura software de un sistema de control
complejo se puede definir como híbrida. Híbrida en el contexto de la arquitectura
software significa que existe en el sistema una mezcla de estilos arquitectónicos que
pueden ser observados mediante la contemplación de una parte u otra del sistema. La
idea es que desde un punto de vista la arquitectura ayuda a gestionar la complejidad
del sistema de control mientras que desde otro u otros puntos de vista la arquitectura
permite controlar la complejidad del desarrollo del sistema software.
También se presenta información al lector sobre los elementos básicos de los sistemas
distribuidos de control. En el mundo real las cosas son más complejas que la simple
cadena sensor, control, actuador y planta. Incluso en ese bucle sencillo existen
elementos especializados dependiendo del área de la industria para la que se diseñe el
sistema de control. El efecto de las comunicaciones en el sistema de control no es
despreciable y la consideración de latencias, retrasos de transmisión o jitter entre otros
introducen nuevos problemas en el diseño del control del proceso. Al mismo tiempo,
el sistema de control incorpora interfases hombre-máquina y software de tipo SCADA
para facilitar a los operadores el seguimiento y supervisión del estado de la planta. En
las capas superiores de la jerarquía de control es posible encontrar desde sistemas de
ejecución del proceso de fabricación (MES), que controlan todos los procesos de
fabricación en tiempo real, hasta ERPs que realizan una tarea parecida a la del MES
sólo que su alcance temporal, al contrario que en el MES, está fijado en el medio y
largo plazo.
Como ejemplo de los elementos básicos que se pueden encontrar en el entorno del
proceso a controlar se detallan varios productos comerciales de utilización en
diferentes ámbitos en los que ICa ha demostrado su utilidad. Se hace especial hincapié
en los Intelligent Electronic Devices (IED) o dispositivos electrónicos inteligentes
como elementos capaces de realizar la función de varios dispositivos en uno solo y
como elemento básico sobre el que establecer un sistema de inteligencia distribuida.
Este tipo de elementos es la base de los grandes sistemas distribuidos de control.
Como contrapunto, también se ofrecen ejemplos de dispositivos que actúan en
sistemas distribuidos y empotrados con características de tiempo real crítico. Por
ejemplo, se muestra el hardware de un elemento para aplicaciones drive-by-wire que
utiliza la tecnología de comunicaciones TTP soportada por ICa.
Como se ha mencionado, el mecanismo de comunicaciones empleado en los sistemas
distribuidos de control tiene gran importancia al no ser despreciables los retardos
inherentes a la red. Dependiendo del tipo de sistema de control distribuido, existirán
diferentes tipos de redes (incluso dentro del mismo sistema) con características
temporales diferentes. En general, cuanto más cerca del proceso a controlar se
encuentre la red, más exigentes serán los requisitos temporales y de otros tipos (por
ejemplo, de tolerancia a fallos). En el ámbito industrial, estas redes encajan
aproximadamente en las distintas capas de la arquitectura del sistema de control
distribuido. Se puede distinguir entre las que se ha venido a denominar red de campo o
fieldbus, red de control y la red de gestión o empresarial. Cada una juega su papel
dentro del sistema de control y se incluye información sobre los protocolos
dominantes en el mercado para cada una de ellas. A su vez, también se considera el
caso de los pequeños sistemas distribuidos (por ejemplo, la red de un automóvil) y los
protocolos disponibles en el ámbito de dicho tipo de sistemas.
La tecnología de los sistemas distribuidos de control se encuentra en constante
evolución y ese hecho afecta a la topología de las redes de control distribuido. El
segundo capítulo incluye información sobre las tendencias pasadas, presentes y futuras
en lo que a topología y distribución de la inteligencia en el sistema de control se
refiere.
En la parte final del capítulo se presentan algunos sistemas que constituyen el estado
del arte en cuanto a control distribuido hoy en día, y que se encuentran disponibles de
manera comercial. Dado el interés de esta Tesis en un amplio rango de sistemas
distribuidos diferentes, se muestran tres sistemas en los que ICa puede ser objeto de
aplicación. El primer sistema se denomina Experion Process Knowledge System de
Honeywell. Es un sistema para grandes plantas que abarca toda la jerarquía de capas
de control. El segundo es un sistema distribuido, empotrado y crítico en cuanto a
tiempo real y en cuanto a seguridad (safety, no security). Se trata del control de
presión de cabina del mayor avión comercial construido hasta la fecha, el Airbus 380.
El tercer ejemplo es diferente de los dos primeros. Se trata del sistema de control
distribuido de las subestaciones de alimentación eléctrica de los trenes de Alta
Velocidad Española que hacen el recorrido Madrid-Barcelona. Uno de los objetivos
primordiales de este sistema de control es el de la alta disponibilidad del mismo en un
entorno geográficamente muy distribuido.
El capítulo finaliza con una crítica de las principales limitaciones de cada uno de los
sistemas expuestos. En el primer ejemplo, el sistema de Honeywell, el principal
problema es el de la dependencia de un fabricante o proveedor que proporciona el
conjunto del sistema. En el segundo ejemplo, el problema es la falta de herramientas
adecuadas para el desarrollo del sistema de control distribuido. Cualquiera que haya
trabajado en sistemas empotrados conoce las limitaciones de las herramientas de
desarrollo. Añádase a esto la dificultad de hacer el sistema distribuido. Finalmente, en
el caso del sistema de control del AVE, el problema proviene de la cantidad de
proveedores diferentes que intervienen en el sistema de control, de los diferentes
protocolos de comunicación empleados y de la dificultad de que el conjunto funcione
con la fiabilidad requerida. ICa proporciona una solución a estos problemas mediante
un marco común de integración que es útil en sistemas de control distribuido de
cualquier tamaño.
El tercer capítulo, Middleware de Componentes Distribuidos, discute los diferentes
modelos de middleware existentes en la actualidad. De una manera sencilla podemos
entender el middleware como el lazo de unión entre todos los elementos de un sistema
que les ayuda a interactuar. El middleware media entra las aplicaciones y la red y el
sistema operativo, aislando a la aplicación de todos los detalles de bajo nivel en la
construcción del sistema. Para conseguir ese aislamiento el middleware realiza una
abstracción para la programación del sistema distribuido con la que todos los módulos
del mismo deben cumplir. El middleware proporciona a la aplicación una interfaz de
programación que permite utilizar los diferentes servicios proporcionados por el
mismo. Algunos de esos servicios son los siguientes:
•
Localización de servicios en la red de una manera transparente. Esto significa
que la distribución del sistema se realiza de una manera transparente al
mismo. Para los componentes del sistema es como si éste fuera monolítico.
•
•
•
•
La aplicación es independiente de los servicios de red. Las aplicaciones no son
conscientes de la red. El middleware se encarga de la gestión de los detalles de
acceso a la misma.
El middleware proporciona fiabilidad y disponibilidad. La programación a
nivel de red es una tarea compleja. El middleware proporciona una manera
probada en miles de aplicaciones de pegar los componentes de una aplicación
distribuida.
El middleware es capaz de ser escalado en capacidad sin pérdida de
funcionalidad. Proporciona servicios que permiten controlar el ciclo de vida
de los componentes del sistema distribuido y controlar los recursos
disponibles para el conjunto del sistema.
Resuelve el problema de la heterogeneidad, permitiendo la construcción de
sistemas donde existe heterogeneidad de hardware, software, sistemas
operativos, protocolos de red y lenguajes de programación.
Se puede realizar una clasificación taxonómica del middleware de acuerdo con las
formas siguientes.
•
•
•
•
Monitores de transacciones (TPM).
Middleware de llamadas a procedimientos remotos (RPC).
Middleware orientado a mensajes (MOM).
Middleware orientado a objetos (OOM u ORB).
La clasificación distingue los tipos de middleware de acuerdo únicamente a los
diferentes mecanismos utilizados en la comunicación entre las partes que componen el
sistema distribuido. En esencia el criterio se basa en que todos los tipos de middleware
definen un esquema uniforme de acceso al resto de los componentes del sistema
distribuido.
El TPM es un middleware de tipo cliente / servidor específicamente diseñado para
aplicaciones transaccionales. Es una tecnología madura, que se utiliza
fundamentalmente en aplicaciones que gestionan datos, bases de datos, emisión de
órdenes (por ejemplo de compra), etc. Este tipo de middleware se diseña para
proporcionar acceso a miles de clientes en un entorno distribuido cliente / servidor
mediante el control y asignación de la peticiones de los clientes a una serie de rutinas
de servicio que multiplexan las transacciones.
Las rutinas de servicio son críticas en esta arquitectura y son las únicas que interactúan
con el sistema servidor evitando a éste tener que realizar la gestión de miles de
peticiones simultáneas. La ventaja es que el sistema servidor sólo ve un pequeño
número de rutinas con las que intercambia información y no los miles de clientes que
en realidad existen. El acceso de los clientes se ordena en función de la prioridad
asignada al servicio requerido. Otra ventaja es que la lógica de la aplicación puede
moverse de los clientes al monitor de transacciones resultando mucho más sencillas
las tareas de administración y actualización de los sistemas. La tecnología TPM es
fácilmente escalable cuando el número de clientes crece, ya que siempre es posible
añadir más monitores de transacciones para gestionar mejor la multiplexación de las
peticiones de los clientes. Además, la mayoría de los productos comerciales
proporcionan facilidades para la gestión del equilibrio de carga o características
avanzadas de gestión y tolerancia a fallos. El mercado se encuentra dominado por
unos pocos fabricantes entre los que se encuentran BEA, IBM o HP.
El middleware tipo RPC es una infraestructura tipo cliente / servidor que permite a los
desarrolladores acceder a los componentes de un sistema distribuido mediante
llamadas a funciones remotas. En este caso, no existe una capa de software que
implementa el middleware. El middleware se encuentra embebido en las aplicaciones
cliente y servidor mediante la inclusión de ficheros stub que se compilan y enlazan
con el resto de la aplicación. Los stubs son invocados por el programa cliente cuando
se intenta llamar a una función que se encuentra en el servidor RPC remoto. Este
mecanismo actúa de forma síncrona aunque existen algunos fabricantes que ofrecen
funcionalidad para realizar llamadas asíncronas.
La ventaja del mecanismo RPC es que enmascara las llamadas a funciones remotas
como si se estuvieran realizando de una manera local. Como las llamadas se realizan
de manera síncrona, en aplicaciones de cierta complejidad es necesario recurrir a la
programación concurrente lo que sin duda hace más complicado el desarrollo de los
sistemas. La mayor crítica o desventaja de los sistemas basados en RPC es que
proporcionan un modelo de programación de relativo bajo nivel enfocado hacia
procedimientos. Existen multitud de situaciones donde la tecnología de objetos
distribuidos proporciona mejores soluciones. El mercado de soluciones RPC se nutre
de dos o tres fabricantes como Entegrity Solutions, Sun o HP.
El middleware orientado a mensajes o MOM se basa en una arquitectura tipo cliente /
servidor que soporta invocaciones asíncronas entre clientes y servidores. En un
sistema basado en MOM los mensajes no se envían directamente entre cliente y
servidor. Existe una cola de mensajes tanto en el lado del cliente como en el del
servidor. El propósito de la cola de mensajes es su almacenamiento en el caso de que
la aplicación que debe procesar los mensajes de la cola se encuentre ocupada
realizando otras tareas. En esta arquitectura, una vez que el emisor ha enviado el
mensaje puede continuar su ejecución. No queda bloqueado esperando respuesta. El
mensaje queda almacenado en la cola de mensajes que intentará enviarlo al programa
receptor hasta que éste lo acepte. Una vez que el receptor construya el mensaje de
respuesta será enviado a su cola de mensajes hasta que el emisor que inició la
comunicación sea capaz de aceptarlo.
La ventaja de los sistemas basados en MOM es que desacopla los procesos cliente y
servidor. Esta era una de las mayores desventajas del mecanismo RPC. Como MOM
permite la interoperación de módulos que son independientes entre sí (se puede enviar
un mensaje a un servidor que no se encuentra operativo en un momento dado) es una
arquitectura bastante apreciada en sistemas de integración empresarial. El mercado se
encuentra controlado por grandes empresas que desarrollan su actividad en el mundo
empresarial. IBM o Software AG se encuentran entre los principales proveedores de
sistemas MOM.
El último tipo de middleware es el orientado a objetos y componentes. Este tipo de
middleware hace especial hincapié en el concepto de interfaz en un entorno de
programación orientado a objetos. Todos los servicios son accedidos a través de una
interfaz y no es necesario conocer ningún detalle de implementación de los mismos. El
middleware orientado a objetos está especialmente pensado para soportar el modelo de
programación orientado a objetos obteniendo las mismas ventajas que tienen estos
sistemas. De esta manera, mediante el middleware orientado a objetos se pueden
construir sistemas que representan bien sistemas del mundo real que son distribuidos
por naturaleza.
El middleware orientado a objetos utiliza un manager orientado a objetos (OOM) o un
broker de peticiones de objetos (ORB) como la capa software que media entre los
objetos local y remoto. Los objetos remotos exponen su interfaz que es accedida de
manera local mediante objetos proxy o representantes de los objetos remotos. El proxy
envía las peticiones al ORB que toma las acciones necesarias para transmitir la
petición del servicio al objeto remoto. Entre las ventajas que aporta este tipo de
middleware se pueden citar las siguientes:
•
•
•
•
•
•
•
No existen clientes ni servidores, sólo objetos que ofrecen servicios y otros
que reciben servicios.
Transparencia en cuanto a la situación del objeto. La localización del
proveedor del servicio es transparente al usuario del servicio.
La distribución de los servicios se puede realizar en cualquier momento.
Los objetos proveedores de servicios se pueden migrar dinámicamente sin
afectar a los usuarios de los mismos.
El middleware introduce el concepto de servicios generales como el servicio
de nombres, de suscripción a eventos o el de gestión del ciclo de vida de los
objetos.
La arquitectura del sistema se hace abierta y escalable.
El middleware hace al sistema independiente de los lenguajes de
programación y del hardware utilizado.
En las actualidad, las tres implementaciones de este tipo de tecnología dominantes en
el mercado son las basadas en la especificación de CORBA del OMG, la tecnología
Java / RMI de Sun MicroSystems y la tecnología DCOM de Microsoft. En este
capítulo se abunda en la manera que cada una de estas tecnologías construye su visión
del middleware orientado a objetos y en una comparación de la complejidad relativa
de cada una de ellas.
El cuarto capítulo se encuentra dedicado a los sistemas basados en la tecnología
CORBA. Introduce los conceptos básicos de la tecnología y las especificaciones más
relevantes para el mundo del control distribuido desarrolladas por el OMG.
El OMG es una organización sin ánimo de lucro fundada en 1989 con el objeto de
desarrollar especificaciones independientes de los fabricantes que permitan fomentar
la utilización de la tecnología de objetos distribuidos. La organización cuenta hoy en
día con más de 500 miembros y mantiene contacto permanente con otras
organizaciones de estandarización como ISO o W3C.
La visión de los sistemas de objetos distribuidos del OMG está basada en la
arquitectura de gestión de objetos u OMA. La OMA especifica cómo construir
sistemas abiertos de objetos distribuidos mediante el uso de un broker de objetos
(ORB) y una colección de servicios predefinidos. La arquitectura se basa en cuatro
componentes que pertenecen a dos tipos en función del servicio que proporcionan:
ORBs y servicios a objetos, y objetos de aplicación y utilidades genéricas. Estos
elementos describen los bloques principales que componen la OMA.
•
•
•
•
•
Gestor o broker de peticiones de objetos (ORB): Es el elemento central de la
arquitectura que permite la transmisión de las peticiones y sus respuestas entre
los objetos CORBA.
Servicios a objetos (CORBAservices): Proporcionan servicios genéricos de
bajo nivel. Son útiles en la práctica totalidad de aplicaciones. Ejemplos de
servicios genéricos son el servicio de nombres o el servicio de notificación.
Utilidades horizontales (CORBAfacilities): Son servicios genéricos de alto
nivel por lo que no pertenecen a la categoría de servicios. Ejemplos son
utilidades de impresión, de correo electrónico o de gestión del tiempo.
Utilidades verticales (Domain CORBAfacilities): Proporcionan servicios
estándar en un campo específico de aplicación (por ejemplo en servicios
sanitarios, telecomunicaciones o transporte).
Objetos de aplicación: Proporcionan un servicio específico dentro de un
sistema dado. Son servicios desarrollados para construir un sistema particular.
Como todas las arquitecturas software, la OMA resulta de un compromiso resultante
de dirimir las tensiones existentes entre los requisitos de la misma: facilidad de
construcción con la tecnología existente, generalidad que permita su construcción por
muchos fabricantes diferentes, transparencia entre aplicaciones cliente y objetos
proveedores de servicios, interoperabilidad entre implementaciones de la arquitectura
y construcción de puentes hacia otras tecnologías, capacidad de evolución,
extensibilidad, etc.
Los servicios CORBA son interesantes en la práctica totalidad de aplicaciones
distribuidas porque proporcionan funcionalidad preconstruida que se puede reutilizar.
Son útiles de manera horizontal a todos los dominios de aplicación. Algunos de los
servicios más interesantes son los siguientes.
•
•
•
•
•
•
•
Tiempo extendido. Es una especificación que extiende el servicio de tiempo
para que sea capaz de representar relojes con diversas características
(resolución, precisión, estabilidad, etc.). Es interesante cuando se utiliza más
de un reloj en un sistema.
Servicio de eventos. Permite desacoplar los objetos que solicitan servicios de
aquellos que los proporcionan. Permite la comunicación asíncrona de eventos
mediante peticiones CORBA.
Servicio de ciclo de vida. Proporciona una interfaz para crear, eliminar, copiar
y mover objetos. Permite a los solicitantes del servicio realizar operaciones de
ciclo de vida sobre los objetos que residen en diferentes localizaciones.
Servicio ligero de registro. Es un servicio que permite la gestión de registros
en aplicaciones de tiempo real o empotradas.
Servicio de nombres. Permite designar a los objetos mediante una notación
comprensible por las personas. Asocia referencias interoperables a objetos con
nombres identificables por las personas.
Servicio de exteriorización. Proporciona un protocolo que permite exteriorizar
e interiorizar objetos. Exteriorizar es registrar el estado de un objeto en una
corriente de datos. Interiorizar es el proceso contrario.
Servicio de estado persistente. Proporciona una interfaz para la retención del
estado de los objetos. Permite restaurar el estado dinámico de un objeto a
partir de su estado persistente. Este servicio se basa en el servicio de
exteriorización.
De las especificaciones de CORBA, la más interesante para los sistemas de control
distribuido es la especificación de CORBA para tiempo real. Esta especificación
extiende CORBA de manera que sea posible controlar los recursos del sistema y así
aumentar la predecibilidad de una aplicación de objetos distribuidos. Existen dos
especificaciones de tiempo real en CORBA. La primera contempla los sistemas
basados en un esquema de prioridades fijas mientras que la segunda soporta la
planificación dinámica del sistema de objetos distribuidos. Dada la complejidad de la
planificación dinámica de un sistema de tiempo real, los ORBs que existen en la
actualidad sólo soportan el mecanismo de planificación basado en prioridades fijas
(salvo alguna excepción experimental).
El objetivo de la especificación de CORBA para tiempo real con prioridades fijas es el
de permitir la construcción de sistemas distribuidos que sean predecibles. En la
especificación definida por el OMG, esto quiere decir que a) las prioridades de los
hilos de ejecución se deben respetar entre clientes y servidores, b) se debe acotar la
inversión de prioridad durante el procesamiento de invocaciones entre objetos y c) se
deben acotar las latencias de las invocaciones a servicios. La especificación se apoya
en extensiones de entidades clave de CORBA (ORB, adaptador de objetos, modelo de
concurrencia, etc.) y de los servicios proporcionados por ellos. La especificación de
tiempo real de CORBA permite gestionar tres tipos de recursos en un sistema
distribuido.
•
•
•
Recursos de procesador.
Recursos de memoria.
Recursos de comunicaciones.
Respecto a la gestión de los recursos utilizados por el procesador, mediante la interfaz
proporcionada por la especificación de CORBA para tiempo real es posible utilizar
mecanismos de gestión para determinar la prioridad con la que se deben ejecutar las
invocaciones CORBA. De la misma manera, los objetos servidores pueden controlar
el número de hilos de ejecución que se utilizarán para resolver peticiones concurrentes
y simultáneamente limitar la prioridad de los mismos. Adicionalmente, el modelo de
control de recursos define un elemento de sincronización que utiliza la misma
semántica que la empleada por los hilos de ejecución para minimizar las inversiones
de prioridad en el sistema.
En cuanto a los mecanismos de la gestión de la prioridad se debe tener en cuenta que
los sistemas que deseamos diseñar son heterogéneos y distribuidos. Por este motivo
CORBA define el concepto de prioridad. Es posible entonces, diseñar el sistema
utilizando prioridades CORBA que a su vez serán relacionadas a través del
middleware con las correspondientes prioridades nativas de cada sistema operativo en
tiempo de ejecución. Para coordinar las prioridades dentro del sistema, la
especificación define dos modelos de prioridad.
•
•
Modelo de prioridad declarada por el servidor. En este modelo la prioridad
que utiliza el servidor para atender una invocación viene definida por el
mismo. La prioridad que se utilizará para atender la petición de un cliente se
encapsula en las referencias a objetos, permitiendo de esta forma que los
clientes conozcan esta información.
Modelo de prioridad declarada por el cliente. Existen ocasiones en las que es
mejor mantener en el lado del servidor la prioridad de ejecución que el cliente
utiliza. Es una manera de evitar las inversiones de prioridad en el camino de
ejecución de la petición. En este modelo, cada invocación o petición al
servidor encapsula la prioridad a utilizar siendo posible que cualquier hilo de
ejecución que se utilice en el lado del servidor se ejecute con la prioridad
deseada por el cliente. La prioridad encapsulada es una prioridad CORBA de
manera que mediante el mapping de prioridad definido por la especificación
pueda ser transformada en una prioridad nativa de cada sistema operativo
utilizado.
En los casos anteriores, el control más fino que se puede obtener respecto a la
prioridad de ejecución es a nivel de objeto pero pueden existir situaciones donde lo
que se necesita es controlar la prioridad a la que se va a ejecutar una determinada
invocación. Para este caso, la especificación define las llamadas transformaciones de
prioridad que permiten cambiar dinámicamente la prioridad a la que se ejecuta una
invocación. Las transformaciones de prioridad son de dos tipos, inbound y outbound.
La primera se ejecuta cuando el middleware del lado servidor recibe una invocación y
antes de que la misma sea despachada al objeto que soporta la operación solicitada.
Las transformaciones outbound ocurren cuando un objeto servidor se encuentra
procesando una operación y tiene a su vez que invocar a otro objeto servidor. En ese
momento se produce una transformación de tipo outbound.
Otro elemento que se puede controlar en el lado del servidor son los hilos de ejecución
empleados para atender las invocaciones de las aplicaciones cliente. Para este
propósito se utiliza el concepto de threadpool o pila de hilos de ejecución. Un
threadpool es una colección de hilos de ejecución que se puede asociar a los
adaptadores de objetos de un servidor CORBA. Mediante la configuración de las
características del threadpool es posible diseñar un servidor concurrente con muy
poco esfuerzo ya que el middleware se encarga de la gestión de todos los hilos de
ejecución.
En la especificación de CORBA para tiempo real existen dos modelos de threadpool,
con y sin carriles. En el modelo sin carriles simplemente se crea una colección de
threads con una serie de propiedades comunes a todos ellos (por ejemplo, prioridad de
ejecución). Los threads se crean estáticamente, lo que permite que estén preparados en
cualquier momento para atender una invocación por parte de un cliente. También es
posible especificar el número de threads que el threadpool puede crear dinámicamente
en caso de que sea necesario (para manejar situaciones en las que se producen
incrementos puntuales de uso del servidor). En el caso de que el threadpool permita la
utilización de carriles, se pueden asignar grupos de thread a cada carril y asignar a
cada carril una prioridad diferente. De esta manera es posible realizar una partición de
la prioridad en función del uso que se le vaya a dar en el sistema a desarrollar. Cada
carril permite también controlar el número de threads estáticos que contiene y el
número de threads dinámicos que se pueden crear. Una última característica de este
tipo de threadpool es que se permite el préstamo de threads entre carrilles. Los
carriles de mayor prioridad pueden promocionar threads de carriles de menor
prioridad a una prioridad superior. Una vez que el thread promocionado ha atendido la
petición para la que se promocionó es devuelto a su carril original.
Para que las aplicaciones construidas sobre el modelo de concurrencia del middleware
sean consistentes es necesario que exista un modelo de sincronización acorde al de
concurrencia. Para esto y como recurso de control del procesador existe una interfaz
para acceder a los objetos mutex que utiliza el ORB. Se asegura así que el mecanismo
de sincronización utilizado por el ORB y la aplicación que se ejecuta sobre él son del
mismo tipo.
El segundo tipo de recurso que se puede controlar es el relativo a la memoria. De cara
a gestionar la misma se puede indicar en el lado del servidor, en el momento de
creación de los threadpool, el buffer de memoria que se desea utilizar para almacenar
las invocaciones que debe atender el threadpool cuando todos sus hilos de ejecución
se encuentran ocupados atendiendo otras invocaciones. Además, se puede configurar
el tamaño del stack que utilizará cada thread para atender las invocaciones de las
aplicaciones cliente.
El tercer tipo de recurso que se puede gestionar son los relacionados con los enlaces
de comunicaciones. Controlando los recursos utilizados para las comunicaciones es
posible configurar los parámetros o propiedades de un determinado protocolo. En
realidad, se pueden configurar dos protocolos diferentes, el utilizado por el ORB
(GIOP en el caso habitual) y el correspondiente al transporte de red (TCP / IP por
defecto). La interfaz permite asignar varios protocolos a un mismo ORB y el orden de
asignación indica el orden de preferencia en el que se usarán los protocolos para
establecer comunicación.
La selección de los protocolos en CORBA se realiza aplicando políticas de control de
la calidad del servicio. Estas políticas se pueden aplicar tanto en el lado del cliente
como del servidor. Cuando la política se aplica en el lado del servidor la información
que contiene se publica en las referencias interoperables de dicho servidor de tal forma
que los clientes son capaces de conocer los protocolos que soporta el servidor. Cuando
la política se aplica en el lado del cliente (la política se puede asignar en la parte del
servidor y se exportará al cliente o puede ser aplicada directamente en el cliente)
permite configurar a éste para seleccionar un protocolo determinado. Además, en el
lado del servidor la política se aplica a nivel del objeto mientras que en el cliente es
posible cambiar dinámicamente en cada invocación el tipo de protocolo a utilizar.
Además de realizar una selección de protocolos, hay otros factores que intervienen en
la predecibilidad extremo a extremo de una aplicación distribuida. Un primer factor
consiste en la determinación del momento en el que se establece la conexión entre el
cliente y el servidor. En el caso más habitual, se establece la conexión en el momento
de realizar la primera invocación por lo que está petición sufre el retardo relacionado
con el proceso de establecimiento de una conexión. En un ORB diseñado para tiempo
real este retardo se puede eliminar estableciendo una conexión de manera explícita
entre el cliente y el servidor antes de la primera invocación. En el caso general, un
ORB no permite controlar el número de conexiones que se establecen entre un cliente
y un servidor y compartirá una o más conexiones entre los clientes de un servidor.
Esta filosofía no es razonable en aplicaciones de tiempo real por lo que la
especificación define políticas para la utilización de conexiones privadas,
multiplexadas o para establecer conexiones que representan bandas de prioridad. El
ORB es responsable en este caso de la transmisión de cada invocación por la conexión
que representa a la banda que contiene la prioridad deseada para la invocación.
Mediante las interfases expuestas de control de recursos del procesador, memoria y
comunicaciones se puede conseguir mejorar la predecibilidad de un sistema basado en
CORBA. Existe una segunda especificación de CORBA para tiempo real que ataca el
problema de la planificación dinámica aunque no existen implementaciones
comerciales debido a la complejidad que representa la construcción de un ORB de este
tipo. La especificación se basa en el concepto de hilo de ejecución distribuible. Este es
un hilo de ejecución que puede traspasar los límites físicos de una máquina
determinada de manera que se conserven las propiedades del mismo a lo largo del
camino de ejecución. El capítulo 4 contiene información más detallada sobre esta
especificación.
Desafortunadamente, la especificación de CORBA para tiempo real se basa en la
especificación de CORBA. Esto quiere decir que el tamaño del ORB de tiempo real es
mayor que el de un simple ORB de CORBA. En sistemas, habituales en control,
donde existen restricciones de memoria, recursos de procesador, etc. esta situación es
claramente prohibitiva. La especificación minimum CORBA proporciona un perfil de
CORBA reducido para este tipo de aplicaciones manteniendo al mismo tiempo la
compatibilidad con la especificación general de CORBA. Esta Tesis tiene en cuenta
estos problemas y presenta un middleware de tiempo real en dos versiones; una
soporta la especificación de CORBA y otra presenta la interfaz de tiempo real sobre la
especificación de minimum CORBA. Es posible entonces tener las ventajas de un
broker CORBA de tiempo real sobre la especificación para sistemas empotrados.
El quinto capítulo desarrolla una de las partes más importantes de esta Tesis y
muestra las características del broker CORBA desarrollado para aplicaciones de
control distribuido. El capítulo presenta al lector las debilidades de CORBA a medida
que su aplicación se realiza más cerca del proceso a controlar ya que los requisitos del
sistema de control son cada vez más exigentes.
El broker CORBA para tiempo real de ICa (Arquitectura de Control Integrado) es el
resultado de un esfuerzo a largo plazo para obtener una infraestructura que permita
desarrollar sistemas de control distribuido de una manera sencilla. Dado el interés de
esta tecnología de cara a los objetivos sociales y económicos de la Unión Europea,
esta iniciativa ha sido extensamente financiada en el marco de los programas ESPRIT
e IST. ICa surgió a partir de la necesidad de reducir el esfuerzo de programación a
nivel de comunicaciones en aplicaciones distribuidas. El primer intento de resolver el
problema, hacia el final de los años 80, dio lugar a la librería CCL (Complex
Communications Layer) que encapsulaba el API de programación de la red a nivel de
TCP/IP y que fue desarrollada durante el proyecto CONEX.
Fue también hacia el final de 80 y principios de los 90 cuando se fundó el OMG y
cuando comenzó a tener gran relevancia la Arquitectura de Gestión de Objetos u
OMA. Nuestro grupo de investigación UPM-ASLab se dio cuenta de la importancia
que la arquitectura tendría en el futuro y decidió construir una capa software sobre la
librería CCL que le permitía seguir la filosofía de desarrollo orientada a objetos. Este
primer ORB ya se denominó ICa pero no era un broker CORBA al no cumplir con la
especificación. Al mismo tiempo, siendo un gran avance en nuestra línea de
investigación incluyendo aspectos como la configuración del marshalling o
proporcionando timeouts programables para las invocaciones, no consideraba aspectos
fundamentales para los sistemas de tiempo real y de control distribuido tales como la
optimización de los algoritmos del ORB o la configuración de la calidad de servicio en
el sistema. También se desarrolló un compilador denominado ADL que utilizaba un
lenguaje de definición de agentes para construir los stubs de comunicaciones que las
aplicaciones necesitaban.
Muchas de las circunstancias que impedían un uso óptimo del ORB en el mundo del
control fueron identificadas y con la financiación obtenida en el proyecto DOTS del
programa europeo IST se construyó un ICa ORB completamente nuevo que se utilizó
con éxito en el área de sistemas de automatización de subestaciones eléctricas. En este
punto ICa ORB ya era un broker CORBA para tiempo real especialmente preparado
para su uso en sistemas industriales. Aún así, los sistemas que se podían atacar eran
aquellos de tiempo real no crítico y el objetivo que se quería alcanzar era el de poder
utilizar ICa en un abanico de sistemas tan amplio como fuera posible. Con este
objetivo en mente se consiguió más financiación y se lanzó el proyecto HRTC
mediante el que se ha dotado a ICa con la capacidad de trabajar en entornos de tiempo
real crítico.
En el caso de ICa se han estudiado las causas de falta de determinismo en el ORB de
manera que se aumente la predecibilidad de ICa ORB. Algunas de las mejoras del
broker son las siguientes.
•
•
•
El código generado automáticamente por el compilador de IDL para los
esqueletos en el lado del servidor y para los stubs en el lado del cliente se ha
mejorado notablemente de manera que el paso de información de las
invocaciones a través de estos adaptadores entre el middleware y el código de
la aplicación sea muy eficiente. Lo mismo ocurre con el encapsulado de las
invocaciones a formato GIOP. ICa ORB utiliza GIOP a nivel de protocolo de
mensajes del ORB para garantizar la interoperabilidad con otros ORBs
comerciales.
Para que un broker sea predecible la gestión de la memoria debe ser
controlada. El uso continuado de memoria dinámica (ciclos de solicitud y
liberación de memoria) es costoso en términos de recursos y puede llevar a
situaciones de fragmentación de la misma. ICa ORB permite la configuración
de los buffers utilizados para el almacenamiento de las invocaciones de
manera que no exista una gestión dinámica de esta memoria.
El transporte por defecto en CORBA es IIOP. Esta transporte esta basado en
TCP/IP que no es predecible. Este problema se ha solventado mediante la
•
•
incorporación de un mecanismo de plug-in de transportes en ICa ORB que
permite la utilización de transportes predecibles.
Despacho de hilos de ejecución. Otras de las partes importantes para evitar la
pérdida de predecibilidad consiste en evitar la inversión de prioridad al asignar
hilos de ejecución para servir invocaciones en el lado del servidor. ICa ORB
incorpora un mecanismo reactivo que permite asignar hilos de ejecución a
invocaciones cuando se detecta actividad en los buffers del plug-in del
transporte de red utilizado.
Despacho de invocaciones. La identificación del objeto servidor que
implementa la operación solicitada debe realizarse a la mayor brevedad, sin
incurrir en retrasos innecesarios. Para este propósito ICa ORB incorpora
mecanismos de búsqueda basados en algoritmos O(1) que proporcionan una
duración de la búsqueda constante en el tiempo.
Las optimizaciones anteriores son de utilidad en cualquier sistema de tiempo real,
crítico o no. Existen otras mejoras que ICa ORB incluye específicamente para
sistemas críticos.
•
•
•
•
En el modelo general de CORBA la interacción entre los objetos distribuidos
se realiza de un modo síncrono. Incluso las invocaciones oneway incorporan
un mecanismo de control de errores que las convierte en bidireccionales y
síncronas. A pesar de ser este mecanismo el mejor de los posibles en cuanto a
la fiabilidad obtenida, la mayoría de las aplicaciones de control (sobre todo las
empotradas) utiliza mecanismos de comunicación no fiable, unidireccional y
no orientado a la conexión. Es posible utilizar llamadas oneway puras en ICa
ORB imitando de esta manera el mecanismo de comunicación utilizado en
muchos sistemas de tiempo real.
ICa puede ser utilizado en sistemas disparados por eventos o en sistemas
disparados por tiempo. Las arquitecturas disparadas por tiempo son más
frecuentes en sistemas de misión crítica y compensan su falta de flexibilidad
exhibiendo un comportamiento más determinista que los sistemas dirigidos
por eventos.
ICa ORB permite obtener información temporal sobre el progreso del sistema.
Para ello define una arquitectura en la que el transporte de comunicaciones es
el encargado de mantener el tiempo del sistema. La gestión del tiempo se ha
incorporado a la interfaz del marco extensible desarrollado para la utilización
de plug-ins de transportes. A su vez, se ha extendido el modelo de objeto
CORBA para acceder a la interfaz del transporte y permitir la obtención de
información relacionada con el progreso temporal de las invocaciones
realizadas en el sistema.
Como trabajo futuro, se propone un modelo de lenguaje de definición de
interfases (IDL) en el que es posible definir las tareas periódicas, aperiódicas y
esporádicas.
•
•
ICa ORB proporciona un servicio de registro del tiempo en el que las
invocaciones se completan mediante políticas aplicables a nivel de objeto.
Finalmente, existen estadísticas sobre los protocolos de transporte utilizados
que permiten conocer información en tiempo de ejecución tales como retardos
máximo, mínimo, medio, etc.
Para demostrar la idoneidad de las mejoras realizadas, durante el desarrollo del
proyecto HRTC se han construido dos transportes de tiempo real crítico. El primero,
basado en RT Ethernet, puede ser utilizado en sistemas disparados por eventos
mientras que el segundo, que utiliza el protocolo TTP, está orientado a sistemas
disparados por tiempo. Ambos transportes han sido utilizados junto con ICa en
aplicaciones empotradas como se muestra en capítulo dedicado a las aplicaciones de
ICa.
El sexto capítulo, presenta una metodología para el desarrollo de sistemas de control
basados en ICa (ICm). El objetivo final de ICa es permitir a los desarrolladores
construir los sistemas de control que imaginen. Para ello, no es suficiente con disponer
de la infraestructura software que ICa ORB proporciona. Es también necesario
disponer de una metodología que ayude a conseguir el comportamiento y
características de tiempo real adecuadas para los sistemas distribuidos de control
desarrollados con ICa. La metodología ICm contiene las líneas maestras del proceso a
seguir para construir sistemas distribuidos siguiendo prácticas de ingeniería. La
metodología propuesta recomienda un proceso de desarrollo en el que se siguen
técnicas de ingeniería de dominio para la creación sistemática de los modelos de
dominio y arquitecturas software de los sistemas de control CORBA.
La metodología se basa en la búsqueda de modelos de referencia que permitan realizar
desarrollos genéricos para un conjunto de sistemas distribuidos de control. Mediante la
traslación del modelo de referencia al conjunto de sistemas y componentes software
que lo implementan se consigue la arquitectura de referencia y a partir de la
construcción de los activos software que componen el sistema se consigue una
arquitectura software para un tipo de sistemas. La idea es obtener líneas de producto
para los sistemas de control distribuido que se puedan construir como familias de
productos. Los sistemas se desarrollan a partir de los activos software que constituyen
el núcleo de la familia de productos.
Para que lo anterior sea posible es necesario que la arquitectura software del sistema
se desarrolle de una manera estándar. La estandardización es útil porque obliga a
elevar el nivel de abstracción y a pensar en las necesidades de múltiples sistemas. Se
debe pensar en las necesidades del dominio de aplicación. Esa es la idea básica de la
ingeniería de dominio. De esta manera, ICa proporciona una infraestructura básica
sobre la que activos software para un determinado dominio de control distribuido
pueden ser construidos. En otras palabras, la metodología incide en la utilización de la
ingeniería de dominio sobre la infraestructura de ICa para la construcción de activos
software que permiten construir conjuntos de sistemas distribuidos de control dentro
de un contexto dado. Por ejemplo, sistemas de control de tráfico aéreo o sistemas de
control de subestaciones eléctricas. El foco debe situarse en el proceso de ingeniería
necesario para desarrollar clases o conjuntos de sistemas distribuidos de control y no
soluciones únicas a problemas de control específicos. Al desarrollarse el trabajo en el
ámbito de los dominios y no de las aplicaciones es necesario en cada momento decidir
el alcance del desarrollo realizado, esto se realiza mediante la técnica de focalización
progresiva del dominio.
La metodología propuesta se basa en el análisis, diseño e implementación de los
elementos del dominio. La utilización de la ingeniería de dominio permite que durante
la fase de análisis se acote el dominio de trabajo y que mediante el estudio de sistemas
preexistentes se obtenga las características comunes y diferentes de los mismos.
Además, el objetivo es obtener una comprensión de las relaciones entre los elementos
del dominio. El análisis da como resultado un modelo del dominio que se utiliza para
realizar el diseño del dominio teniendo en cuenta la utilización de ICa para el diseño
de una arquitectura genérica de la familia de sistemas de control basada en los
resultados del análisis y en las clasificaciones existentes de patrones de sistemas y
estilos arquitectónicos. La fase de implementación del dominio consiste en la
elaboración de repositorios de componentes (este término se usa aquí de forma
genérica) reutilizables para el dominio de control. Los componentes se han
identificado a partir de la arquitectura desarrollada durante las fases de análisis y
diseño del dominio.
De esta manera, y mediante la aplicación de la técnica de focalización progresiva del
dominio, se construyen diferentes soluciones o activos software (denominados
componentes en el párrafo anterior) que se utilizan en la construcción de una familia
de sistemas distribuidos de control CORBA. Esos activos se denominan objetos ICa,
patrones ICa, activos ICa y subsistemas o sistemas según su grado creciente de
complejidad. Un sistema de control concreto se construye a partir de la composición y
particularización de los componentes descritos para un problema de control
determinado.
Para facilitar la utilización de la metodología descrita en el Capítulo 6, cada una de las
fases de la ingeniería de dominio para los sistemas de control CORBA proporciona los
pasos que se deben seguir en el proceso de desarrollo. La ventaja es que la
metodología siempre cuenta con la utilización de ICa ORB como infraestructura
básica de manera que gran parte del trabajo a realizar en cada fase ya se encuentra
implementado. A lo largo del capítulo, se demuestran los pasos a seguir mediante la
presentación de un ejemplo real de la utilización de la metodología consistente en la
construcción de la infraestructura necesaria para la construcción de una línea de
productos como una familia de productos para la automatización de subestaciones
eléctricas.
La fase de análisis del dominio de los sistemas de control CORBA consiste en la
determinación del alcance e identificación del dominio de control, el análisis
sistemático de las características comunes y de la variabilidad de la familia de
sistemas del dominio de control. Además, incluye la construcción de un conjunto de
requisitos para la familia de sistemas. A pesar de que la metodología da preferencia a
la construcción de soluciones de ingeniería de dominio para ICa, es posible tomar en
la fase de análisis la perspectiva de la ingeniería de aplicación. El resultado de la
aplicación de la metodología desde este punto de vista da lugar a la construcción de
soluciones específicas de control distribuido. Es decir, da lugar a la construcción de un
sistema de control distribuido.
El objetivo del diseño del dominio de los sistemas de control CORBA es tomar los
resultados del análisis de dominio realizado con ICm y producir un diseño e
implementación genéricos para una familia de sistemas. Para ello es necesario
mantener el nivel de abstracción utilizado durante la fase de análisis (en el sentido de
que no se piensa en un solo sistema sino en múltiples sistemas). El resultado del
diseño es una arquitectura software basada en ICa que es reutilizable en múltiples
sistemas. El resultado del proceso de diseño para el dominio proporciona una
estrategia de partición del dominio que define los componentes de la arquitectura y la
asignación del conjunto de características del dominio a cada uno de ellos. Además, se
obtiene un modelo de control que define el modelo de comportamiento de los
elementos software que constituyen la arquitectura. La construcción de la
infraestructura para la familia de sistemas de control del dominio (de los componentes
software de la familia) se apoya sobre ICa, ya que ésta es una infraestructura de
infraestructuras o lo que es lo mismo, ICa se ha construido realizando ingeniería de
dominio para el dominio de todos los sistemas de control distribuido.
La implementación de los sistemas de control CORBA se apoya en las fases previas
de la metodología, el análisis y diseño del domino. Estas fases anteriores concluyen
con la construcción de una infraestructura que puede ser reutilizada en todos los
sistemas del dominio y que permite obtener garantías respecto a los atributos de
calidad de los sistemas construidos, sobre todo cuando se compara con lo que se
podría obtener al construir los sistemas mediante la simple utilización de ingeniería de
aplicación. En función de los resultados obtenidos en las fases previas a la
implementación, la construcción de sistemas de control CORBA consiste en gran
medida en la selección y configuración de las características adecuadas de la
infraestructura desarrollada para el dominio. El objetivo de la metodología es la
producción de la menor cantidad de código e implementación ad-hoc como sea
posible. En cualquier caso, será necesaria la identificación de las características del
sistema no implementadas por la infraestructura del dominio y, o bien su
incorporación a la infraestructura básica del dominio, o bien su construcción mediante
ingeniería de aplicación. La infraestructura del dominio se refina en un proceso
iterativo mediante la construcción de nuevos sistemas de control a partir de la misma.
Finalmente, el capítulo relativo a la metodología concluye con algunas valoraciones
sobre los resultados obtenidos mediante la metodología propuesta en comparación con
los sistemas de producción que se encuentran en uso en el dominio de la
automatización de subestaciones eléctricas. Los resultados obtenidos permiten afirmar
que el uso de la metodología, del proceso de desarrollo, basado en la arquitectura
software y en ICa ORB presenta numerosas ventajas en comparación con el desarrollo
típico de una aplicación de control.
El séptimo capítulo presenta algunos ejemplos de sistemas de control CORBA
desarrollados durante los trabajos de esta Tesis. Uno de los objetivos de la Tesis
consistía en la demostración de que una única tecnología podía ser capaz de servir
para la construcción de sistemas dispares de control. Desde grandes sistemas
distribuidos a pequeños sistemas empotrados con necesidades de comunicación. Este
capítulo presenta un resumen de la financiación obtenida para el desarrollo de estos
trabajos así como los sistemas construidos que responden a dominios diferentes dentro
del dominio más amplio de los sistemas de control distribuido (focalización progresiva
del dominio).
En general, la financiación de los proyectos de investigación se ha obtenido de la
Unión Europea a través de los programas IST y ESPRIT. Los proyectos financiados
han sido los siguientes.
•
•
•
•
ESPRIT Project 22130 - Distributed Information Technology for Strategic
Multiobjective Process Control (DIXIT).
IST-1999-10251 - Distributed Object Telecontrol Systems and Networks
(DOTS).
IST-2001-37652 - Hard Real-Time CORBA (HRTC).
IST-004669 – COMPonent Approach for Real-time and Embedded
(COMPARE ). Este proyecto continua los trabajos de esta Tesis.
Cada uno de los proyectos explora diferentes aspectos en el uso de tecnología
CORBA. El proyecto DIXIT es el más antiguo y se centra en la interoperabilidad en
sistemas heterogéneos para conseguir el control de la planta desde el punto de vista
estratégico. Utiliza diferentes técnicas de inteligencia artificial (sistemas fuzzy, redes
neuronales, sistemas expertos) para conseguir realizar predicciones sobre el estado de
la planta y actuar en consecuencia. El proyecto DOTS se centra en objetivos de
telecontrol de sistemas a través de redes. CORBA es el elemento integrador y se
estudia la construcción de sistemas a partir de un modelo general del dominio de
control. El proyecto HRTC lleva ICa ORB un poco más allá, al acometer su
utilización en entornos de tiempo real crítico tanto para grandes sistemas distribuidos
como para sistemas empotrados.
El resto del capítulo presenta una variedad de sistemas ejemplares construidos con la
infraestructura de ICa y su metodología. Existen ejemplos en muchos dominios en los
que existen diferentes problemas de control. Una breve descripción de cada uno de
ellos es la siguiente.
•
•
•
•
•
•
Control Distribuido de Procesos. Es un sistema para realizar el control de una
planta química piloto. El objetivo fundamental del sistema es la identificación
de los requisitos necesarios en este tipo de sistemas y la realización de
experimentos en condiciones de heterogeneidad y de integración de sistemas
propietarios.
Control Distribuido de un Robot Industrial. El objetivo de este sistema es la
demostración de las deficiencias temporales observables en el cálculo del
control distribuido mediante la utilización de ICa para tiempo real y de un
ORB en el que no se contemple el tiempo real. El objetivo práctico consiste en
la captura de manera autónoma de una pelota que una persona lanza al brazo
robótico (un manipulador ABB IRB2000). Para ello, se utiliza visión artificial
estereoscópica para la determinación de la trayectoria de la pelota en tiempo
real.
Sistema de Telecontrol de Subestaciones Eléctricas. Otro de los sistemas
desarrollados contempla el control distribuido de subestaciones eléctricas. El
objetivo es la implementación de servicios estándar para la construcción de
este tipo de sistemas y la obtención de interoperabilidad mediante ICa para
dispositivos de control de distintos fabricantes.
Hydra-Visión. Es un sistema que proporciona vídeo digital a los operadores de
la sala de control en una red de área extensa. El sistema se ha utilizado para
tener supervisión directa de las acciones de control en plantas hidráulicas de
generación de energía eléctrica. Consta de una serie de nodos que generan
vídeo y audio digital en formato MPEG y una serie de nodos que consumen el
audio y vídeo generados. Las cámaras y micrófonos se pueden controlar de
manera remota e incluso se pueden definir blancos para las cámaras o realizar
la grabación de cualquier suceso que ocurra en la planta. Un protocolo de
interoperabilidad con el SCADA permite el posicionamiento y grabación
automática de cualquier evento en la planta controlada.
Teleoperación de un Robot Virtual. El objetivo de este proyecto consiste en la
demostración de técnicas de objetos distribuidos. Mediante el uso de ICa es
posible definir una interfaz genérica y construir un sistema distribuido que
permita la utilización de un sistema robótico de cualquier tipo mediante un
joystick con realimentación de fuerza, una interfaz gráfica de usuario, una
página web o cualquier otra forma de acceso a los servicios del sistema.
RiskMan. Es un gestor de riesgos (Risk Manager) para plantas petroquímicas.
El sistema realiza la supervisión de los elementos que puedan afectar a la
seguridad y al mantenimiento de la planta. Respecto a la seguridad, el objetivo
del sistema es la prevención de situaciones anómalas, informando en todo
momento al personal adecuado de la planta y lanzando el plan de emergencia
de la misma en caso necesario. Respecto al objetivo de mantenimiento,
RiskMan analiza el impacto de las operaciones de mantenimiento de la planta
•
antes de su realización. Este análisis es necesario ya que la combinación de
diversas operaciones de manteniento sobre la planta de manera simultánea en
la misma área geográfica puede ser peligrosa. La tecnología que da soporte a
la arquitectura de RiskMan es ICa.
PIKMAK. El acrónimo significa Información del Proceso y Modelado del
Conocimiento para Control Avanzado (de Process Information and
Knowledge Modeling for Advanced Control). Los objetivos de PIKMAK se
centran en el control de la calidad, cantidad y mantenimiento del proceso de
producción de clinker. El objetivo de cantidad se atacó mediante la
construcción del PPSI (Production Performance Synthetic Indicator) o
Indicador Sintético del Rendimiento de la Producción). Respecto al objetivo
de calidad el problema principal es el control de algunos parámetros que
afectan al proceso de producción. Para su control se desarrolló el QDED
(Quality Deviation Early Detector) o Detector Temprano de la Desviación de
Calidad. El objetivo de mantenimiento consiste en la asistencia a los
operadores durante los turnos nocturnos de fin se semana en los que controla
la planta un solo operador. Para este propósito se desarrollo el AMOA (Alarm
Manager Operator Assistant) o Gestor de Alarmas Asistente del Operador.
AMOA asiste al operador en la toma de decisiones sobre si un fallo en la
planta debe ser o no reparado inmediatamente. Aunque esta decisión puede
parecer sencilla, en realidad existen numerosos factores a tomar en cuenta ya
que el punto óptimo de operación de la planta depende del perfil de coste de la
energía eléctrica para las horas y días siguientes a la detección del problema,
del stock existente de clinker, del plan de producción, de la disponibilidad de
repuestos para las reparaciones, etc.
El octavo capítulo presenta las conclusiones y trabajos futuros a realizar en el entorno
de ICa y, en particular, de ICa ORB. Como se explica, los objetivos fundamentales de
la Tesis han sido completamente satisfechos. El primer objetivo consistía en la
demostración de una tecnología e infraestructura que permitiese acometer con
facilidad el desarrollo de sistemas de control distribuido en una variedad de dominios
y que garantizase las cualidades finales del sistema final. Para ello, se ha desarrollado
y utilizado el broker ICa de tiempo real tanto en grandes sistemas industriales como
para sistemas empotrados distribuidos con diferentes requisitos en cuanto al
comportamiento de tiempo real. El segundo objetivo fundamental consistía en el
desarrollo de una metodología que permita la aplicación de ICa para la construcción
de Sistemas de Control CORBA de una manera sistemática (o más bien metódica). La
metodología ICm adopta el punto de vista de la Ingeniería de Dominio lo que
introduce la posibilidad de o bien la construcción de sistemas específicos de control
sobre ICa mediante la utilización de Ingeniería de Aplicación, o bien la construcción
de entornos específicos (también sobre ICa) para el desarrollo de múltiples sistemas
(de familias) de control CORBA en un dominio específico de control. La elección de
una u otra alternativa depende de los objetivos del equipo de desarrollo.
Entre las conclusiones de la Tesis, existen algunas que son el resultado de la evolución
de los trabajos a lo largo de estos últimos años y que no hemos llamado fundamentales
ya que son un producto de estas últimas. Las ventajas que el uso de la tecnología
CORBA tiene para el desarrollo de sistemas de control no serían aprovechables por
otros si no se realiza una labor de divulgación de las posibilidades de la tecnología.
Durante los últimos años esta labor de divulgación ha sido una constante. El principal
resultado de este esfuerzo ha sido la creación dentro del subgrupo de Sistemas de
Tiempo Real, Empotrados y Especializados del Object Management Group (OMG) de
un grupo para la especificación de interfases para Sistemas de Control CORBA. Este
grupo de denomina Control Systems Working Group. La creación de especificaciones
dentro de este grupo garantiza el consenso en su creación, de manera que se defiende
el uso de estándares para la construcción de estos sistemas, otro de los objetivos de
esta Tesis.
Otro de los resultados de los trabajos realizados durante los últimos años ha sido el
desarrollo de la infraestructura necesaria para la construcción de Sistemas de
Automatización de Subestaciones eléctricas mediante la aplicación de la metodología
ICm de esta Tesis. La mencionada infraestructura corresponde a la ingeniería de
dominio contenida en el estándar IEC-61850 y al conocimiento del mismo de
compañías del sector. Esta infraestructura es la primera implementación a nivel
mundial del estándar sobre tecnología CORBA (ICa).
También existen terceros que ya se han beneficiado del conocimiento sobre CORBA
obtenido durante de los trabajos realizados en esta Tesis. En este sentido, he recibido
peticiones de diversas empresas para proporcionar asesoría como experto en
tecnología CORBA. De esta manera, he podido asesorar a compañías de diferentes
ámbitos tales como el entorno militar para algunos sistemas de la O.T.A.N., el entorno
de la gestión del tráfico aéreo para el desarrollo e interoperabilidad de los futuros
sistemas del Cielo Único Europeo o el entorno del control de líneas de producción
mediante tecnología CORBA. El interés de estas compañías por la utilización de esta
tecnología para los propósitos de control no es más que una corroboración de los
resultados aquí expuestos. En este sentido, la Tesis representa una contribución de
valor añadido a las políticas de la Unión Europea respecto a la estandarización,
interoperabilidad de sistemas y mejora de las condiciones de vida de los europeos
mediante la mejora del desarrollo de los sistemas de control distribuido.
Por supuesto, el trabajo realizado abre una serie de posibilidades que es necesario
explorar como parte de los trabajos futuros a desarrollar sobre los resultados de la
Tesis. El primero de estos trabajos está relacionado con el desarrollo de servicios para
Sistemas de Control CORBA. La disponibilidad de servicios predefinidos permitiría el
uso de ICa de manera que se pueda seleccionar funcionalidad preconstruida para
sistemas de control distribuido. El resultado de esta aproximación es una manera más
predecible de construir sistemas de control así como la estandarización de su
desarrollo. Los servicios pueden ser de muchos tipos y, en muchos casos, útiles para
muchos dominios de control: servidores de datos, gestores de alarmas y eventos,
servicios de históricos de datos, servicios de diagnóstico, servicios de análisis y
generación de informes, servicios de integración de dispositivos, servicios para
interfases de usuario, etc.
El segundo de los trabajos futuros a realizar sería la construcción de infraestructuras
sobre ICa para el desarrollo de sistemas distribuidos de control en dominios
específicos. La idea es usar ICa ORB e ICm para construir infraestructuras de la
misma manera que se ha hecho para los SAS. Mientras que los servicios del párrafo
anterior tienen validez para muchos dominios de control diferentes, las
infraestructuras sirven para desarrollar sistemas de control en un solo dominio.
El tercer trabajo futuro a realizar sobre ICa sería la construcción de un Entorno de
Control Integrado (ICe) para ICa. Actualmente, la mayoría de las herramientas de ICa
están desarrolladas para su utilización desde la línea de comandos. La inclusión de
entornos gráficos para el desarrollo de sistemas de control con ICa y el uso de técnicas
de desarrollo rápido de aplicaciones (RAD) facilitarían mucho las tareas relacionadas
con el trabajo y uso diario de ICa. Esta alternativa se está explorando por el grupo de
investigación dentro del marco del entorno Eclipse y el modelo de desarrollo MDD
(Model Driven Development).
Finalmente, como conclusión a los trabajos futuros, se encuentran las tareas
fundamentales de mantenimiento y evolución de ICa ORB. Se debe explorar la
inclusión en ICa ORB de funcionalidad para el soporte de planificación dinámica de
tareas. La idoneidad (en la práctica) de funcionalidad de este tipo está por determinar.
También es necesario desarrollar diferentes plug-ins para soportar los protocolos de
comunicación más utilizados en la industria. Esta puede ser una labor específica de
cada dominio en muchos casos por lo que es posible que esos plug-ins sean parte del
desarrollo de la infraestructura para un dominio. Por último, otra labor rutinaria pero
de obligada realización es la migración de ICa ORB a cuantas plataformas y sistemas
operativos sea posible. Para ser realmente interoperable hay que ser independiente del
hardware y del sistema operativo. ICa ORB está perfectamente preparado para ser
migrado a cualquier plataforma gracias a la capa de portabilidad existente en su
arquitectura.
Abstract
Many control systems of today are heterogeneous Distributed Control Systems (DCS).
They are also complex systems as a great number of software and hardware elements
are composed together to perform global functions at the plant level. The number of
reasons for system distribution is wide: the control computation must be close to the
process as communications delays can not be afforded, the plant is in itself distributed
in a geographical area, there is a need of reliability and fault tolerance for the control
system and many others. These reasons, that become requirements, make distributed
control systems difficult to build and the software developed for them keeps
increasing its costs while, at the same time, their hardware costs are decreasing.
This Thesis presents a research effort to address these problems of construction of
distributed control systems. The Thesis advocates the use of a common technology
(CORBA) specialized for distributed control systems in a wide span of distributed
control domains. Thus, CORBA is the base upon which heterogeneity, complexity and
lack of system integration can be tackled in the development of DCSs.
For the research, a real-time CORBA Object Request Broker, ICa ORB, has been
completely rebuilt from a previous research work in the Integrated Control
Architecture (ICa) which can be denominated a long-term research effort. There is not
a single line of code from the first version of ICa in its new release, so it is a fully new
ORB and not just a revamp of the previous one. ICa ORB has been built according to
the CORBA, minimum CORBA and real-time CORBA specifications as it is
considered in this Thesis that open specifications and standards are needed in order to
provide full integration among disparate systems. ICa has been devised as the
common technology that is suitable for the development of either large control
systems or small ones like embedded distributed systems with hard real-time
requirements. In order to demonstrate the feasibility of the approach, to use a single
technology for the development of DCSs in many different control domains, a wide
range of control systems has been developed in the past years. Substation Automation
Systems (SAS), strategic control of petrochemical plants, strategic control of cement
production plants or robot distributed control systems are some of them.
Along with the ICa ORB development effort, a methodology for its use in the
construction of DCSs is also presented in this Thesis. The ICm methodology uses a
domain engineering approach to develop control systems as product lines from the
core software assets of a product family. Domain engineering is used to infer and
develop those core software assets for distributed control. The assets constitute a
software framework that allows developing multiple control systems for a certain
control domain (e.g. substation automation systems or automotive applications) or just
one-of-a-kind systems for a specific control purpose.
ICa ORB and ICm are used to develop distributed control systems in a process that
makes development problems of DCSs disappear or at least be greatly alleviated.
When a distributed control system is built in this way it is said in this Thesis to be a
CORBA Control System, which is the rationale behind its title.
Table Of Contents
Chapter 1
Introduction ..........................................................................................1
1.1
Introduction ....................................................................................................3
1.2
Software Architecture.....................................................................................5
1.3
Object Technology .........................................................................................6
1.4
Distributed Object Frameworks......................................................................9
1.5
Distributed Control Systems.........................................................................11
1.5.1
Execution Environment.........................................................................12
1.5.2
Complexity ............................................................................................14
1.5.3
Real-Time Features ..............................................................................15
1.6
CORBA Control Systems.............................................................................15
1.7
Need of Integration.......................................................................................17
1.8
The Importance of Standards........................................................................18
1.9
Thesis Objectives..........................................................................................18
1.10 Thesis Structure ............................................................................................19
Chapter 2
Industrial Control Systems and Protocols........................................21
2.1
Distributed Process Control..........................................................................23
2.1.1
Types of Distributed Systems to Control ..............................................24
2.1.2
Layers of Control..................................................................................25
2.1.3
Process Control Elements ....................................................................27
2.1.4
Sensors, Controllers, Actuators............................................................28
2.1.5
Industrial Communications Protocols..................................................31
2.1.6
Trends of Control Networks .................................................................34
2.2
Commercial State Of The Art DCSs ............................................................35
2.2.1
Honeywell Experion PKS .....................................................................35
2.2.2
Airbus 380 Cabin Pressure Control System .........................................37
2.2.3
AVE Power Distributed Control System...............................................38
2.3
Critique Statement ........................................................................................41
Chapter 3
Distributed Component Middleware ................................................43
3.1
Definition of Middleware .............................................................................45
3.2
Taxonomy of Middleware ............................................................................46
3.2.1
Transaction-Processing Monitor (TPM) Middleware..........................46
3.2.2
Remote Procedure Call (RPC) .............................................................48
i
3.2.3
Message Oriented Middleware (MOM)................................................50
3.2.4
Object-Oriented and Component Middleware .....................................51
3.3
Components vs. Distributed Objects ............................................................54
3.4
Open Group’s Distributed Computing Environment....................................55
3.5
OMG’s Distributed CORBA Architecture ...................................................57
3.6
Microsoft’s Distributed Component Architecture ........................................59
3.6.1
.NET Framework ..................................................................................61
3.6.2
OLE for Process Control ......................................................................62
3.6.3
Microsoft’s View of DCOM and CORBA .............................................63
3.7
Sun’s Java Distributed Component Architecture .........................................64
3.8
Comparing Apples to Apples .......................................................................67
Chapter 4
CORBA Systems .................................................................................69
4.1
CORBA Technology ....................................................................................71
4.2
The OMG......................................................................................................71
4.3
Brokering Architectures ...............................................................................72
4.4
Object Management Architecture.................................................................73
4.5
Services and Facilities ..................................................................................75
4.6
Real-Time CORBA ......................................................................................78
4.6.1
Fixed-priority Real-Time CORBA ........................................................79
4.6.2
RT-ORB ................................................................................................80
4.6.3
Real-time POA......................................................................................80
4.6.4
CORBA Priorities .................................................................................81
4.6.5
Real-time Current Interface..................................................................82
4.6.6
Real-time CORBA Priority Models and Transforms ............................82
4.6.7
Synchronization ....................................................................................84
4.6.8
Handling Concurrency .........................................................................84
4.6.9
Handling of Connections ......................................................................86
4.6.10 Invocation Timeout ...............................................................................88
4.6.11 Protocol Configuration.........................................................................89
4.6.12 Real-Time Scheduling Service ..............................................................90
4.7
Dynamic-Scheduling Real-Time CORBA ...................................................90
4.7.1
Distributable Thread ............................................................................91
4.7.2
Real-time CORBA Scheduler................................................................93
4.7.3
Advice about Real-Time CORBA..........................................................94
4.8
Minimum CORBA .......................................................................................94
4.9
Fault-Tolerant CORBA ................................................................................95
4.9.1
Grouping Abstractions .........................................................................95
4.9.2
Architectural Overview.........................................................................97
4.10 Data Distribution ..........................................................................................98
4.11 CORBA Component Model .........................................................................99
4.11.1
What is a CORBA Component?..........................................................100
ii
4.11.2
Limitations of the CCM for Distributed Control ................................101
4.11.3
The COMPonent Approach for Real-time and Embedded (COMPARE)
Project 102
Chapter 5
Using CORBA in Control Systems..................................................105
5.1
CORBA Weaknesses..................................................................................107
5.2
CORBA Control Systems Domains ...........................................................110
5.3
The Integrated Control Architecture (ICa) ORB ........................................110
5.4
Levels of Requirements ..............................................................................112
5.5
Sources of Non - Determinism ...................................................................114
5.6
General ICa ORB Performance Improvements ..........................................116
5.6.1
Client Stubs and Server Skeletons ......................................................117
5.6.2
Marshalling ........................................................................................117
5.6.3
Memory Management .........................................................................118
5.6.4
Transport Buffering and Delays .........................................................118
5.6.5
Thread Dispatching ............................................................................119
5.6.6
Request Dispatching ...........................................................................119
5.7
ICa ORB Specific Features for Hard Real-Time Systems..........................122
5.8
CORBA Communication Model ................................................................122
5.9
CORBA Control Loop................................................................................123
5.9.1
CORBA Controllers............................................................................124
5.9.2
Timing Constraints .............................................................................126
5.9.3
Loop Timing........................................................................................127
5.10 Event Triggered vs. Time Triggered Systems ............................................128
5.11 Distributed Synchronization .......................................................................129
5.11.1
Clock Driven / Time Triggered Systems .............................................130
5.11.2
RT Ethernet / Event Driven Systems...................................................130
5.12 The Notion of Time ....................................................................................130
5.12.1
Driving the System from the Communications Layer .........................131
5.12.2
Interface Extensions to the Extensible Transport Framework ...........131
5.12.3
Where to Ask for the Time in the Pluggable Framework? .................132
5.12.4
Particularization for the ETF Submission..........................................134
5.12.5
Asking the Time from an ICa Application ..........................................137
5.12.6
Life as a RTObject ..............................................................................138
5.13 Periodic, Aperiodic and Sporadic Activities ..............................................139
5.14 Timestamping .............................................................................................141
5.15 HRTC Protocol Statistics............................................................................142
5.16 C++ Source Code Example ........................................................................143
5.17 The TTP Transport Plug-in.........................................................................145
5.17.1
Main Constraint for a HRTC TTP Transport Plug-in ........................145
5.17.2
GIOP Messages Written into CNI ......................................................145
5.17.3
Pluggable Data Interface ...................................................................147
iii
5.17.4
Active Server.......................................................................................148
5.17.5
Standard CORBA + TTP/C Application.............................................149
5.17.6
Implemented Option for TTP/C ..........................................................150
5.18 The RT-Ethernet Transport Plug-in............................................................151
5.18.1
Switched Scheduled Ethernet..............................................................152
5.18.2
Real-Time Channels ...........................................................................153
5.18.3
RT Ethernet Pluggable Transport ......................................................153
5.19 Platforms for CORBA Control Systems.....................................................155
5.19.1
Embedded industrial PC Boards ........................................................155
5.19.2
Networked Device Servers..................................................................155
5.19.3
Control Units ......................................................................................156
5.19.4
Time-Triggered Hardware .................................................................156
5.20 The ICa CORBA IED.................................................................................157
Chapter 6
A Methodology for CORBA Control Systems ...............................161
6.1
Need of a Methodology ..............................................................................163
6.2
Domain Engineering for CORBA Control Systems ...................................164
6.3
Methodology Overview..............................................................................167
6.4
CORBA Control Systems Domain Analysis ..............................................169
6.4.1
CORBA Control Systems Scope..........................................................169
6.4.2
ICa Domain Analysis..........................................................................170
6.4.3
Steps of CORBA Control Systems Domain Analysis with ICa............172
6.4.4
Example of ICa Domain Analysis for SAS..........................................174
6.5
CORBA Control Systems Domain Design.................................................177
6.5.1
ICa Domain Design and Implementation ...........................................177
6.5.2
Steps of CORBA Control Systems Domain Design and Implementation
with ICa 178
6.5.3
Example of ICa Domain Design for SAS............................................180
6.6
CORBA Control Systems Implementation.................................................185
6.6.1
Steps of CORBA Control Systems Implementation with ICa..............186
6.6.2
Example of ICa Control Implementation for a SAS Application ........187
6.7
Pattern and Architecture Exploitation ........................................................191
Chapter 7
Exemplar CORBA Control Systems...............................................197
7.1
Summary List of Funding for this Work ....................................................199
7.1.1
IST-2001-37652 Hard Real-Time CORBA (HRTC) ...........................199
7.1.2
IST-1999-10251 Distributed Object Telecontrol Systems and Networks
(DOTS) 200
7.1.3
ESPRIT Project 22130 – Distributed Information Technology for
Strategic Multiobjective Process Control (DIXIT).............................................202
7.2
Exemplar Applications of ICa ....................................................................203
7.2.1
Process Control Testbed.....................................................................204
7.2.2
Distributed Robot Control ..................................................................206
iv
7.2.3
Substation Telecontrol Application ....................................................208
7.2.4
HYDRA-Vision....................................................................................211
7.2.5
Teleoperation of a Virtual Robot........................................................215
7.2.6
RISKMAN ...........................................................................................218
7.2.7
PIKMAC .............................................................................................219
Chapter 8
Conclusions and Future Work ........................................................223
8.1
Conclusions ................................................................................................225
8.1.1
Specific Thesis Objectives ..................................................................225
8.1.2
Awareness of Standardization Bodies ................................................226
8.1.3
Domain Engineering of Substation Automation Systems ...................227
8.1.4
Results for Third Parties.....................................................................227
8.1.5
Research Support................................................................................228
8.2
Future Work................................................................................................229
8.2.1
CORBA Control System Services .......................................................229
8.2.2
Development of Domain Specific Control Frameworks .....................231
8.2.3
The ICa IDE .......................................................................................231
8.2.4
Extending and Maintaining ICa ORB.................................................232
8.2.5
ICa Pattern Language ........................................................................233
GLOSSARY ..............................................................................................................235
Bibliography..............................................................................................................265
v
List Of Figures
FIGURE 1: A COMPLEX PROCESS PLANT..........................................................................4
FIGURE 2: A CONTROL OBJECTS HIERARCHY IN ICA ......................................................8
FIGURE 3: CONTROL LAYERS .......................................................................................13
FIGURE 4: HYBRID ARCHITECTURE OF COMPLEX CONTROLLERS .................................26
FIGURE 5: A MULTIFUNCTION IED FROM GE ...............................................................29
FIGURE 6: A TTP DRIVE-BY-WIRE BOX FROM TTTECH................................................30
FIGURE 7: A SNOW GROOMER BASED ON A TTP NETWORK (FIGURE AUTHORING OF
TTTECH) ..............................................................................................................30
FIGURE 8: GEOGRAPHICAL DISTRIBUTION OF FIELD BUS USAGE ..................................31
FIGURE 9: PROCESS PLANT INTEGRATION ....................................................................32
FIGURE 10: TDMA BUS ACCESS SCHEME IN TTP.........................................................33
FIGURE 11: TRENDS OF CONTROL SYSTEM NETWORKS ................................................35
FIGURE 12: PLATFORM ARCHITECTURE OF EXPERION PKS (FIGURE AUTHORING
BELONGS TO HONEYWELL) ..................................................................................36
FIGURE 13: AIRBUS 380 CABIN PRESSURE CONTROL SYSTEM SCHEMA ........................38
FIGURE 14: AVE POWER SUBSTATIONS DISTRIBUTED CONTROL SYSTEM (FIGURE
AUTHORING BELONGS TO AELEC) ......................................................................39
FIGURE 15: TYPES OF MIDDLEWARE SERVICES ............................................................46
FIGURE 16: TRANSACTION PROCESSING MONITOR ......................................................47
FIGURE 17: REMOTE PROCEDURE CALL MIDDLEWARE ...............................................49
FIGURE 18: MOM ARCHITECTURE ...............................................................................51
FIGURE 19: GENERAL ARCHITECTURE FOR OBJECT-ORIENTED MIDDLEWARE .............52
FIGURE 20: CLASSIFICATION AND TYPES OF COMPONENTS..........................................55
FIGURE 21: COM COMPONENTS RUNNING IN INTRA-HOST PROCESSES........................59
FIGURE 22: DCOM INTER-HOST ARCHITECTURE .........................................................60
FIGURE 23: THE I / O DRIVER DATA ACCESS PROBLEM ................................................62
FIGURE 24: RMI ARCHITECTURE ..................................................................................65
FIGURE 25: THE J2EE ARCHITECTURE .........................................................................66
FIGURE 26: THE CORBA OMA....................................................................................74
FIGURE 27: REAL-TIME CORBA EXTENSIONS PROVIDE STRONG CONTROL OVER
APPLICATION RESOURCES ON THE CLIENT AND SERVER SIDES ............................79
FIGURE 28: CORBA PRIORITIES AND MAPPING TO OS ................................................81
vii
FIGURE 29: CLIENT PROPAGATED PRIORITY MODEL ....................................................83
FIGURE 30: SERVER DECLARED PRIORITY MODEL ........................................................83
FIGURE 31: THREADPOOL MODELS, WITH AND WITHOUT LANES .................................85
FIGURE 32: ASSIGNMENT OF THREADPOOLS TO REAL-TIME POAS ..............................86
FIGURE 33: PRIORITY BANDS AND EXPLICIT BINDING ..................................................87
FIGURE 34: MULTIPLEXED AND PRIVATE CONNECTIONS IN REAL-TIME CORBA .......88
FIGURE 35: PROTOCOL CONFIGURATION IN REAL-TIME CORBA ................................89
FIGURE 36: CONTROL FLOW IN A DISTRIBUTED PROCESSING SYSTEM .........................91
FIGURE 37: DISTRIBUTABLE THREAD WITH NESTED SCHEDULING SEGMENTS .............92
FIGURE 38: FAULT TOLERANCE DOMAINS ....................................................................96
FIGURE 39: ARCHITECTURAL OVERVIEW OF A FAULT TOLERANT CORBA SYSTEM ....97
FIGURE 40: EXTENDED CORBA COMPONENT AND PORTS ........................................101
FIGURE 41: AREAS OF WORK FOR THE RT/E COMPONENT FRAMEWORK ..................103
FIGURE 42: A TYPICAL AUTOMATION INFRASTRUCTURE ...........................................107
FIGURE 43: CORBA AIR-TRAFFIC FLIGHT CONTROL SYSTEM (ATC SYSTEM
STRUCTURE FROM [LIU 00]) ..............................................................................113
FIGURE 44: POINTS OF PREDICTABILITY IMPROVEMENT ............................................115
FIGURE 45: RESOURCE CONTROL FEATURES IN ICA ...................................................116
FIGURE 46: COMPARISON OF N AND LOG N VERSUS N ................................................121
FIGURE 47: DISTRIBUTED CORBA CONTROL LOOP ..................................................124
FIGURE 48: EMBEDDED CORBA CONTROL LOOP......................................................125
FIGURE 49: CONSTRAINTS OF LOOP TIMING ...............................................................127
FIGURE 50: THE ICA OCI TRANSPORT PLUG-IN ARCHITECTURE ................................133
FIGURE 51: SOURCE OF TIME INFORMATION IN ICA ...................................................134
FIGURE 52: GIOP MESSAGES COPIED DIRECTLY TO CNI MEMORY ............................146
FIGURE 53: PLUGGABLE DATA INTERFACE .................................................................147
FIGURE 54: THE RT-ETHERNET APPROACH................................................................154
FIGURE 55: A PC104 AND A HDD3.5'' FORM FACTOR INDUSTRIAL PC BOARDS (FROM
LANNER ELECTRONICS) .....................................................................................155
FIGURE 56: AXIS COMMUNICATIONS NETWORK DEVICE SERVER (FRONT AND BACK
SIDES) .................................................................................................................156
FIGURE 57: A CONTROL UNIT FROM ELIOP ...............................................................157
FIGURE 58: A TTP DEVELOPMENT CLUSTER (LEFT) AND A TTP X-BY-WIRE BOX
(RIGHT) FROM TTTECH ......................................................................................157
FIGURE 59: UML DEPLOYMENT DIAGRAM OF THE ICA IED ......................................158
FIGURE 60: UML PACKAGE DIAGRAM OF THE ICA IED.............................................160
FIGURE 61: SYSTEM ARCHITECTURE BUSINESS CYCLE ..............................................164
FIGURE 62: INTEGRATED CONTROL METHODOLOGY PROCESS ..................................168
FIGURE 63: DOMAIN ENGINEERING AND ANALYSIS WITH ICA ...................................171
FIGURE 64: PHASES OF ICA DOMAIN ANALYSIS .........................................................174
FIGURE 65: GENERAL VIEW OF THE SAS APPLICATION DOMAIN. ..............................175
viii
FIGURE 66: GENERAL STRUCTURE OF A SUBSTATION AUTOMATION SYSTEM AS SHOWN
IN THE IEC 61850 STANDARD [IEC 01C]. LEVEL 0 = PROCESS, LEVEL 1 = BAY,
LEVEL 2 = STATION. ...........................................................................................176
FIGURE 67: PHASES OF ICA DOMAIN DESIGN AND IMPLEMENTATION........................181
FIGURE 68: SAS LEVEL 1 SUBSYSTEM IDENTIFICATION ............................................182
FIGURE 69: THE ACSI IMPLEMENTATION FROM IEC-61850......................................184
FIGURE 70: PHASES OF CORBA CONTROL SYSTEMS IMPLEMENTATION ..................187
FIGURE 71: THE REE SUBSTATION IN VILLAVICIOSA DE ODÓN ................................189
FIGURE 72: PHYSICAL LAYOUT FOR INTERLOCKING TESTS ........................................192
FIGURE 73: INTERLOCKING OPERATION REJECTED.....................................................193
FIGURE 74: INTERLOCKING OPERATION PERMITTED ..................................................194
FIGURE 75: HARDWARE SETUP FOR THE PROCESS CONTROL TESTBED ......................204
FIGURE 76: THE PROCESS CONTROL DEMONSTRATOR................................................206
FIGURE 77: ROBOT CONTROL TESTBED SETUP ...........................................................207
FIGURE 78: THE IRB-2000 ROBOT READY TO CATCH THE THROWN BALL. NOTICE THE
TWO CAMERAS ON THE WALL. ...........................................................................208
FIGURE 79: IEC-61850 STANDARD PERSPECTIVE ON FUNCTION PROVISION BASED ON
LOGICAL NODES .................................................................................................209
FIGURE 80: SETUP OF THE DOTS PILOT SYSTEM........................................................211
FIGURE 81: HYDRA-VISION GRAPHICAL USER INTERFACE .........................................212
FIGURE 82: CONTROLLED EQUIPMENT AND HYDRA-VISION NETWORK ....................214
FIGURE 83: THE SIMULATED ANTHROPOMORPHIC ROBOT ARM .................................216
FIGURE 84: MULTIPLE SYSTEMS CAN BE TELEOPERATED WITH A SINGLE IDL
INTERFACE .........................................................................................................217
FIGURE 85: RISKMAN USER INTERFACE...................................................................219
FIGURE 86: PIKMAC USER INTERFACE ......................................................................220
FIGURE 87: CONTROL SYSTEMS WG WEB PAGE AT OMG'S SITE ...............................226
ix
List Of Tables
TABLE 1: GENERAL SYSTEM QUALITY ATTRIBUTES ACHIEVABLE BY ARCHITECTURE 14
TABLE 2: CLASSIFICATION OF MIDDLEWARE FEATURES ..............................................67
TABLE 3: REAL-TIME CORBA PRIORITY MODELS ......................................................82
xi
xii
Chapter 1
Introduction
1
1.1 Introduction
Nowadays, most control systems are heterogeneous distributed systems. They are also
complex systems as a great number of different hardware and software elements are
combined together to provide global functions. The technical reason for heterogeneity
is that specialized equipment performs better some functions of a system; sensorreading and actuator control, process management and enterprise management are
placed at different levels of the distributed control system and perform different tasks.
In the case of control systems, the main reason for distribution is that computing must
be close to the process in order to ensure prompt reaction to process changes (e.g.
lower-level control loops). Figure 1 shows this situation in a complex process plant.
However, there are many other reasons for doing this:
•
availability of suitable embeddable processors,
•
timing requirements that forbid communication due to latencies,
•
need of increased levels of performance that is achieved through parallelism,
•
simplification of construction and maintenance tasks through modularity,
•
reduction of cost and time-to-market by means of component-based reuse,
•
integration of legacy systems
•
availability of task specific software platforms
•
Reliability and fault tolerance.
The example of the process plant can be easily translated to other domains where
distribution and heterogeneity are key factors. The reader may think of modern
warfare systems where information flows from many different sources (radar, satellite,
targetting, engine control data, etc.), and is drained by a similar amount of sinks
(troops, avionics, combat systems, positioning systems, etc). Another example is in the
automotive industry where modern ABS systems have a dedicated computer in each
wheel exchange braking information with the rest. Additionally the suspension, power
management, and engine control systems work together with the ABS to keep the
automobile under control. While these systems are common these days, what are the
common aspects among all of them?
They are difficult to build. While hardware costs keep decreasing over the years, the
software that runs on it becomes more and more complex and the effort and capital
investment to build it keeps rising. The problem has several facets two of which,
3
distribution and heterogeneity, are of surmounting importance in order to properly
build the system.
Safety
MIS
Enterprise Network
Business Management
Data Storage
Process Control
Process Operation
Control Network
Process Management
Field Configuration
Fieldbus
Sensing and Acting
Field Management
Continuous Process Plant
Figure 1: A complex process plant
Embodied in this document, a research on a common technology (CORBA) to address
these problems in the field of distributed control systems is carried out. Control
systems greatly vary from the very small to the very large and their requirements also
vary greatly from the very time-tight systems to others where time is less critical like
those related to the Management Information System of the plant. Generally speaking,
the best way to get rid of those problems is to make them disappear at the system
development level. A common abstraction for a wide variety of systems is needed.
Making the abstraction common means that it is platform-independent, so perceived
heterogeneity can be actually removed. To avoid the complication emerging from
distribution we should develop the system as if it were a monolithic system. These are
precisely the ideas behind the Object Management Architecture (OMA): to provide
abstraction above low-level detailed platform-problems and to provide distribution
and heterogeneity transparency to systems in an object-oriented way. This is achieved
by providing concrete object models based on the OMA; namely CORBA. Concrete
implementations of the CORBA specification hand to system builders a homogeneous
platform-independent infrastructure for system development.
4
But whereas the homogeneous CORBA abstraction has proven useful regarding some
domains, the concrete object model of the CORBA architecture is not good enough in
some situations. CORBA was designed with the objectives of flexibility,
interoperability and reduction of complexity without incurring in a significant loss of
performance. Unfortunately, there are system demands, like those of control systems,
and requirements, as those related to real-time, embedded, and fault tolerant systems,
that cannot be overlooked to build a successfully working system. A careful analysis
and implementation of the supporting framework technology for these systems is then
needed as it must bear the basic properties the final system must exhibit. To meet all
these requirements, a disciplined approach is needed and this Thesis relies in three
areas of computing science to achieve them.
•
•
•
Software Architecture: As the guiding discipline to achieve system
properties that are architectural in nature.
Object Technology: As the technical support for building abstract
representations of the world.
Distributed Object Technology: To provide the basic infrastructure upon
which control architectures are implemented.
1.2 Software Architecture
As distributed control systems are software intensive artifacts, a way to specify their
architecture in the same way other engineering disciplines do is needed. By means of
Software Architecture, critical properties of the final system are defined, which will be
almost impossible to modify in later stages of development.
If control systems were small or if their properties as software system were easy to
achieve we would not need to worry about Software Architecture. Coding algorithms
is probably all we would need to do. Although this might seem a “light-headed”
statement, it is not. All of us know examples of this type of development.
Unfortunately, distributed control systems are neither small nor their requirements are
easy to achieve. When systems begin to grow, there is no other way to success than to
divide the functionality of the system. In other words, we need to define modules that
form part of the system. Defining modules and the relationships between them will
also provide us with the ability of modifying them independently from the rest. When
the problem is that the features are hard to achieve, we also need architecture to be
able to focus in the design of specific parts of the system. For instance, how to design
a module with a fast cyclic digital input scan.
5
Software Architecture is important because it appears early in the life cycle of the
system and helps to make decisions about its composition, structure and the
relationships between all the elements that compose the system. These decisions do
not only affect the properties of the system but, also affect the capability of the
developing organization in its market niche. Software Architecture influences how a
company can compete in the market or what the time-to-market for a family of
products will be. This means that the business organization must not only focus on the
technical properties of a system’s Software Architecture but also in others that affect
how business in done. The software engineering team must then create a balance or
define a trade-off between the qualities of the system from both points of view,
technical and those related to business. In fact, and only from the technical point of
view, defining a Software Architecture will help to make the trade-off about the
technical capabilities of the system; availability, dependability, performance, etc. All
these are qualities of the system that can only be specified by architecture. If an
architecture fails to meet any of the architecture-related attributes of the system, then a
successful system cannot be built. Hence, the importance of enforcing the use of
sound architectures for Complex Control Systems.
1.3 Object Technology
The control engineering community is still reluctant to a change in the core
technology that drives their systems. Understandably, they build and operate control
systems of great complexity, whose requirements are difficult to meet, and in
consequence they stick to technologies that have proven to solve most of their
problems over decades now. Nevertheless, everybody perceives the value and the
advantages that Object Technology (OT) has to offer, precisely in an area whose
properties of complexity and heterogeneity can be suitably addressed by it.
Objects can be thought of as abstract representations of real-world or other entities
that are independent and able to manage by themselves. Functionality is achieved by
the services the objects provide and information is exchanged by message passing. A
first key-point in the object concept is that they encapsulate and hide their state which
can only be accessed or modified by the services provided by the object. Thus,
internally modifying an object does not affect other objects in the system which is the
cornerstone for isolating and managing change in the system. Other objects can
request the services of any other object in order to make the system progress towards
its objective.
6
The second key-point for objects is the concept of abstraction. Through objects we can
capture the essential properties of any system entities, in an attempt of making things
plainly understandable. Deciding which entities are represented by objects and which
are not often becomes a tough decision which will affect the overall system properties.
Further, some objects might just be data-structures while others may provide
computation services only, as to provide only behavior to the system. At the end, the
level of abstraction depends on the type of applications being developed. The real
benefit is that we have representations of notions our minds can easily manage. These
two key-points fit well into the Software Architecture concept of division of
functionality and lower the concepts closer to the implementation of the system. This
is why Object Technology is so interesting in this research. It allows us to translate
Software Architecture concepts to the implementation field.
This apparently simple concept of objects, gives system developers a huge advantage
over the way systems can be constructed. As mentioned before, objects are
independent entities and as such can be understood as standalone parts of a system.
This provides a way to handle complexity in a divide and conquer style or in an
approach to system decomposition. It also provides ground for reusability, allowing
developers to build code with a long life expectancy. Reusability is the best known
way to cut down development costs and to shorten the time needed to build production
systems. For automation industries, the possibility of reusing objects as part of their
systems redirects the organization of their development core processes to develop
software in product lines. This way of building systems allows the developers to share
a common architecture for a range of similar systems which bear a set of common
features and are built from a collection of shared software assets. The companies that
evolve software product lines have largely optimized the way their software control
systems are constructed. The costly investment made in system architecture
development pays off as the same architecture can be reused multiple times. They
have a library of software assets that can be employed many times in a set of products
that share common characteristics. This is important as common control systems are
expected to lend their service over decades, not to mention the effort and capital
needed to build them.
Objects can be mapped to real-world entities that make full sense to human beings.
Further, inheritance can be employed to build hierarchies of objects of increasing
complexity in a step-by-step approach (not iterative). This is just another way to deal
with complexity and heterogeneity. Child objects provide slightly specialized behavior
so the increase of complexity for each generation in the hierarchy is not too large to be
understood. Complex behavior can be obtained from ascendants by creating
descendant objects with different properties, giving birth to hierarchies of more
refined or specialized objects. Figure 2 shows a hierarchy of objects to provide
7
complex control behavior built from simpler objects. Notice than in the figure, there is
an orange shadow on the outer rim of each circle that represents a control object. This
shadowed area is the object’s interface and it is the only way the services of the object
can be requested from other objects. As will be explained later, objects can expose
different types of interfaces that can be classified according to the type of service they
provide.
One of the activities the control systems industry is facing is the shift to object
technology. They have always claimed not to use object technology for a matter of
efficiency. Although this reason may still hold for some systems which must be
developed in machine code, its number is greatly inferior to the remainder of systems
that can benefit from object technology. One of the reasons of the availability of this
change is the evolution of hardware performance. The market for hardware devices
offers ever increasing performance and features for devices are always decreasing in
cost.
Education is another factor that is obliging
manufacturers to make the transition to object
technology. It is becoming harder to find
qualified personnel that knows anything else
but object technology as schools and
Universities worldwide mostly provide
teaching in this area.
ICa Core
Agent
ICa MT
Agent
ICa KB
Agent
ICa RT
Agent
ICa FT
Agent
ICa FL
Agent
ICa PLC
Agent
But objects alone cannot provide the kind of
ICa Fuzzy
ICa RiskMan
service control systems need. Objects need to
Control
Agent
Agent
be combined to provide the service at the
global level in a distributed way and this must
be done according to the set of requirements
control systems need. Composing distributed
objects ad-hoc cannot be considered a sound
practice as the middleware code will be deeply
hooked to the application. For the next system
to develop, new code will need to be written to
achieve the same functions the previous
middleware did. This practice poses all kinds Figure 2: A control objects
of problems to developers as they always hierarchy in ICa
perform the same tasks but in a way particular
to every system implementation. They are not
8
making good use of the gained knowledge or at least they are not optimizing it.
Instead of this approach, a middleware infrastructure based on standards can be
employed to achieve the distribution of objects.
1.4 Distributed Object Frameworks
According to Enslow [Ens 78], distributed systems are characterized by the following
properties,
•
•
•
•
•
They are composed of a collection of computational resources (hardware and
software), which can be dynamically assigned to specific tasks.
The resources are physically distributed, and collaborate by means of a
communication network.
An operating system exists and is used to unify and integrate the control of the
components.
The distribution of the computational resources is transparent, and services
may be requested specifying only their name. It is not necessary to know the
location of the resource, just how it is named.
The operation of the hardware and software resources is characterized by a
coordinated autonomy.
This definition by Enslow in 1978 was one of the first ever done for distributed
systems. There are other definitions which have been formulated later but the content
of the definition is basically the same. For Tanenbaum [Tanenbaum 95], a distributed
system is a collection of independent computers that appear to the users of the system
as a single computer. Coulouris [CDK 94] gives a more detailed definition of what a
distributed system is.
“A distributed system consists of a collection of autonomous computers linked by a
computer network and equipped with distributed system software. Distributed system
software enables computers to coordinate their activities and to share the resources of
the system -- hardware, software, and data. Users of a well-designed distributed
system should perceive a single, integrated computing facility even though it may be
implemented by many computers in different locations”.
In a broad sense, an intuitive yet powerful definition is that of Burns [Burns 97],
9
“A distributed computer system is defined to be a system of multiple autonomous
processing elements, cooperating in a common purpose to achieve a common goal”.
For our work and in the CORBA Control Systems point of view, this notion of
distributed system provides a clear common understanding of what a distributed
control system is, a collection of elements collaborating to run a plant, with certain
real-time properties, efficiently. As to the requirements demanded from distributed
systems they are much the same of any other system. We would like our system to be
efficient, flexible, robust, scalable, dependable, etc. While these requirements are
common requirements for most systems, in the case of distributed systems they are
harder to achieve. The reason for this is that distribution raises the complexity of the
system. The distributed system architecture has to account for the communications
system to be used that will affect all the properties of the system. Think for example in
the real-time properties that must be met and in the latencies that a certain
communication mechanism will impose by its nature onto the system. This is a
problem that would not need to be treated if the system were a monolithic one.
To ease the pain of developing the underlying architecture’s support for distribution,
distributed object frameworks are relied on to implement this task. One of the primary
objectives of this Thesis is to effectively demonstrate that there is a common
technology that can be employed to develop distributed control systems at all levels;
from the embedded system level where decisions are made in milliseconds to the
strategic level where production planning is scheduled at a much larger time
granularity (weeks, months).
A distributed object framework is valuable because it is a common abstraction for
functionality needed in all types of distributed systems. This means we can reuse a
specific implementation of that abstraction in multiple systems, providing leverage to
the work of constructing the infrastructure needed for distribution. The question is
whether there is or can be constructed such a framework that enables us to implement
all layers of a complex control distributed system.
The framework used for the development of CORBA Control Systems should be
object-oriented. There are other types of frameworks for distributed systems such as
the message passing or transaction processing frameworks which are not well fit for
the control purpose. The fundamental reason for which an object approach is needed is
software complexity. Complexity is the effect of human being limitations. A
distributed object-oriented framework helps us solve the problem of complexity in
three ways.
10
•
•
•
Object-oriented frameworks support the principle of abstraction.
Object oriented frameworks support the principle of hierarchical
development.
Object oriented frameworks support the principle of modularity or
decomposition.
An object oriented framework supports abstraction by decoupling the interface of the
object from the implementation of the object. This concept must not be compared with
that of abstract types in the C++ language (similar but no the same) not the abstract
keyword in OMG’s IDL. Through hierarchical development we can distinguish
between the general properties of things and the properties of a specific kind of things.
For instance, all controllers have the general properties of reading inputs and
calculating outputs but only Fuzzy controllers implement fuzzy control algorithms. By
inheritance we can make commonality explicit and increase system functionality in a
simple manner. Decomposition and modularity are inherently supported by
Distributed Object Frameworks as systems specifications are built from a composition
of objects.
1.5 Distributed Control Systems
Distributed Control Systems are a special case of distributed systems by its own
nature. They have very special properties, take heterogeneity for example. Although
heterogeneity can be controlled in some distributed systems (e.g. a banking system), in
the case of distributed control systems heterogeneity is almost impossible to limit. A
large distributed control system implements thousands of functions and is built around
equipment from multiple vendors. Further, this type of systems is designed to service
the plant for long lifetimes (15 years or more) and obviously technology changes over
time. This results in the introduction of new heterogeneity as in the lifetime of the
plant some systems will be replaced by newer ones while others will simply be
maintained. If the complexity of building a heterogeneous distributed system is
considered along with the complexity of building a large control system, a
development problem of overwhelming complexity is posed to system builders.
Distributed Object Frameworks such as CORBA are very effective in dealing with the
problem of heterogeneity, making transparent for developers the hardware or software
nature of the components of a distributed system. However, there are other types of
requirements that are dealt with in an ad-hoc way at the time being. These
requirements are those associated to the real-time properties of the control system. If
11
an object framework is used for the construction of distributed control systems then
the framework must provide the necessary tools to control its real-time behavior so the
whole application can be deployed coherently. In this Thesis, some adaptations of the
current real-time CORBA specification have been used to take control over sources of
non-determinism in the CORBA object framework.
To fixate the scope of CORBA control systems, the execution environment and the
real-time and complexity characteristics of a controlled plant are presented below.
1.5.1
Execution Environment
Distributed Control Systems have a very specific execution environment. The objective
of the DCS is the efficient exploitation of the interoperation capabilities of devices and
systems in the distributed context of the plant. In this distributed scenario, there is a
variety of systems than can be classified by diverse criteria. From a control point of
view, a valid classification can be based on the following items.
•
•
•
functionality
location
performance
For instance and returning to the example of the control layers of a process plant (see
Figure 3), at the lower control level of the plant some Remote Terminal Units (RTU)
or Programmable Logical Controllers (PLC) are devoted to real-time signal
acquisition from the Intelligent Electronic Devices (IED) connected to them. A second
level of RTUs acts as a communications front-end in a multi-layer networked
structure. Both types of remote terminal units can be geographically scattered across
different unattended locations. SCADA systems are located in control centres, far
away from the origin of process signals and do not require the hard real-time
capabilities of lower control layers. There are other tools such as configuration
applications that can be run off-line. Above these layers, the Enterprise Management
System can schedule production for the next days, weeks or months. All the layers are
connected and information flows upward, downward and horizontally through systems
of different vendors that use different technology. Horizontal and vertical integration
is attained at a high cost.
As already mentioned, a common characteristic in most industrial systems is the
heterogeneity of hardware and software in the system. From the point of view of
hardware, proprietary buses, such as VME or Compact PCI systems, are present in the
12
different devices of the plant. Software also presents a high degree of heterogeneity.
Different communication protocols, proprietary monitors for embedded systems and
software from different suppliers (i.e. SCADA software) are easily found in this type
of systems. Almost always, there is also the need to integrate new software with
legacy systems.
User Interface
MIS
Strategic Control
Optimization
Tactical Control
Plan execution
Operational Control
Reactivity
Advanced Control
Complex Loops
Simple Loops
Sensors & Actuators
Conventional
Process
Control
Continuous Process Plant
Figure 3: Control Layers
Thus, the main characteristic of the execution environment is heterogeneity and
complexity. Concerning software there is heterogeneity and complexity in software
architecture. This is because there is not such a thing as a software architecture for the
whole system. These conditions lead to complex systems. Distributed control systems
are complex not only because of their size; they are also complex because of their
nature. The domain of the problem must be considered as well as its size. The first
solution to complexity is decomposition. Split the problem into pieces until we can
understand the complexity in our own minds. Unfortunately, this leads to
heterogeneous systems if not handled with care. Other constraints such as the lack of
standards adoption (proprietary solutions are a business barrier for competitors) make
it impossible to plan a homogeneous system.
Decomposition increases another form of complexity in the system. The isolated
modules must be interconnected and information must be transmitted by new channels
of communication. Communication protocols must be specified and checks for their
13
consistency must be established. This is where CORBA can put some leverage to the
current scenario. The architectural framework of the OMG’s OMA helps reduce this
type of complexity by establishing detailed interface specifications and a common
understanding of what interoperability means.
Additionally, in the case of industrial systems appears the need to control the
timeliness of the system and to embed the software in limited-resource systems. In the
case of real-time features, CORBA can also help dealing with these issues by
facilitating the way in which the lower level real-time OS APIs are accessed. For
embedded systems, support comes from the profiling of CORBA for smaller systems.
It has been a long time since either control systems where based on “island CPUs” or
were conceived as “islands of automation”. Distributed control systems must show
qualities such as performance, reliability, availability, scalability, portability and
maintainability among others (see Table 1). The lack of these characteristics cannot
be afforded in industrial systems but cost effectiveness is also an issue. CORBA as a
standard for Distributed Object Computing contributes to system development by
isolating the developers from the boundaries of system modules.
Quality attribute
Architectural Architectural issues
in nature?
Discerned observing the system at runtime
Yes
Divide functionality to exploit parallelism
Performance
Yes
Fault tolerance with redundant components
Availability
No
Interaction with other quality attributes
Functionality
Not discerned observing the system at runtime
Yes
Modularized components
Modifiability
Yes
Portability layers
Portability
Yes
Loose coupling among components
Reusability
Yes
Consistent component interfaces. Possibility
Integrability
of incremental build
Yes
Modularized components
Testability
Table 1: General system quality attributes achievable by architecture
1.5.2
Complexity
Complexity is important because it affects the way we understand the devised
systems. In the case of distributed control, it is of utmost importance as it affects the
14
robustness of the system. Whereas cost is important, distributed control systems can
produce potentially hazardous effects on life or property if they misbehave. Then, it
happens often than performance is sacrificed instead of robustness. Complexity is
tackled by means of modularity, structuring the system into pieces than enable us to
cope with system’s complexity and focus into the detailed analysis of independent
modules. A technology that helps developers deal with complexity is needed by
approaching system construction in a structured or modular way.
1.5.3
Real-Time Features
Distributed control systems present conflicting requirements. This is due to the fact
that on one side the system is split into modules which make the system bulky but
easier to understand. On the other side, there are the real-time control requirements
which demand an efficient implementation of the system (with regard to
performance). Depending on the time constants of the controlled system, the degree of
efficiency of the implementation may vary. In the case the control is very close to the
process where resources are low, the introduction of infrastructures that allow
modularity may hamper performance as the system needs to execute more code with
the same processing resources. The real-time requirements pose an additional problem
to distributed control systems development as tools for system infrastructure need to
support real-time requirements and must be capable of being deployed over devices
with little processing resources.
1.6 CORBA Control Systems
Distributed objects are a useful technology for control systems construction because
control systems are software systems that continuously interact with the reality (i.e.
with real, physical objects) so the software paradigm that best fits this domain is the
object-oriented paradigm.
The CORBA community has been aware for a long time of the lack of suitability of
the general CORBA specification for certain types of technical systems. They have
also kept in mind the tremendous benefits that CORBA has brought to distributed
system development. This change in the way systems are built has led engineers to
think that the same concepts of abstraction and homogeneity could be applied to
systems with more stringent operating conditions. These are basically systems that
need to deal with the progression of time, dependability or scarce resources.
15
The efforts of the OMG community regarding the Distributed, Real-Time and
Embedded (DRE) systems have been materialized in three different specifications
which now form part of the CORBA specification suite.
•
•
•
Minimum CORBA specification. This is a profile of CORBA for lowresource systems. Basically, it removes from the CORBA specification those
parts which are of little use to systems where most things are known at designtime.
Real-Time CORBA specification. This specification builds on top of
CORBA to provide the control of resources necessary to achieve end-to-end
predictability in real-time systems. The specification reuses concepts and
some parts of other specifications as the Quality of Service framework from
the Messaging specification or the Enhanced View of Time specification.
Fault-Tolerant CORBA specification. There are many applications that
have a need for fault-tolerance. This specification deals with the CORBA
infrastructure services that an application might request to achieve fault
tolerance. The specification supports a range of fault tolerant strategies such
as request retry, redirection to an alternative server, and passive and active
replication.
The increasing interest of the scientific / technical developer community in CORBA
for technical systems have fostered a rising industry of ORB manufacturers for
industrial applications. Most implementations are Real-Time CORBA ORBs built on
top of the Minimum CORBA specification so as to take advantage of real-time
features with a moderate use of resources.
There are some issues that have not still been fully addressed as some requirements
mix-up in critical systems. These systems have requirements regarding real-time,
fault-tolerance and embedded characteristics that pose a difficult problem still
unsolved unless treated ad-hoc by traditional systems engineering. The OMG
specifications regarding real-time, embedded, and fault tolerant systems have been
done individually and whereas most real-time ORBs provide minimum CORBA
implementations they do not provide reliability characteristics regarding faulttolerance. This means that DRE applications can be fully built and deployed by the
use of commercially available ORBs but the task of building the system becomes
more problematic when requirements for predictability and dependability begin to be
tangled together. This is also the situation in the case there is a decision to build the
system in an ad-hoc way, so in benefit of CORBA the developer can always take the
advantage of having a common abstraction for the development of DRE systems no
matter the platform used. Other questions as the lack of awareness of the progression
16
of time in the CORBA abstraction (IDL language) are beginning to be dealt with so in
a future, all types of systems with hard real-time constraints may come into operation
using CORBA technology.
Although, as reviewed, some problems still remain unsolved, the technology has
established the quality of its value in a great extent of industrial applications ranging
from onboard ship command and control or research facilities to software defined
radio, tactical radio systems, telecom applications or avionics. CORBA has clear
advantages regarding transparency (object location), independency (from platforms
and languages), scalability, flexibility, cost, etc. when compared to traditional system
development procedures that can be fully applied to systems with tightly-binding
requirements.
1.7 Need of Integration
When designing distributed control systems, the requirement for integration is always
present. As of today, integration is a costly process that is solved by the introduction
of legacy bridges among communication protocols or by gateways among standard
protocols in the best of cases.
Take for instance, the railway traction substation case. In a small substation, catenary
overhead line protection devices may use MODBUS or the IEC60870-5-103 protocol
to carry out serial communication from the protection devices to bay modules. The
bay modules are attached to a ring-like redundant fiber optics network that uses a
proprietary TDMA-based protocol to communicate all bay modules in the substation
with an industrial PC. The PC is used as an OPC server which can be accessed from
an operator workstation (an OPC client) via a TCP/IP network. Another PC linked to
the TCP/IP network, acts as a gateway to connect the substation to a remote control
center by means of the IEC60870-5-101 protocol. This is common case of a
distributed control system.
Integration is needed either horizontally or vertically. The exposed case only shows
and example of what happens in the lower layers of Figure 3. Most of the protocols
used in the example are employed to achieve horizontal integration but the overall
system also includes production management and business operation levels so vertical
integration is also needed.
In this work, it is shown that CORBA is a technology that, being good in the upper
enterprise management layers of the distributed control systems, can also be employed
17
at the lower control layers to integrate the diverse controllers and devices spread
throughout the controlled process site. Process information can be obtained from any
place in the controlled site and actions can be initiated to improve production with a
single integration technology based upon the OMG specifications.
1.8 The Importance of Standards
Many people do not realize that the world will not be the same without standards.
Standards are one of the forces that have led the way of technology development until
the present day and that have configured modern economy. In other words, standards
shape our world. It all began about one hundred and fifty years ago when William
Sellers gave his speech “On a uniform system of screw threads” in the lecture hall of
the Franklin Institute, the professional society to which Sellers belonged. Before this,
tools were custom-made for the clients. With standardization mass production became
possible, ensuring at the same time that products had an acceptable quality level.
Standardization has become today a political struggle where the strongest stakeholders
try to impose their technologies (an example of this is the Profibus technology backed
by Siemens). Many companies have realized that the best way to develop their
business is to help to create standards so they actively collaborate in the
standardization bodies. One of these bodies is the Object Management Group and one
of OMG’s specifications is CORBA. OMG is the largest object software organization
in the world with more that 500 company members. CORBA is a specification that has
become an open de-facto standard which provides developers of distributed systems
with a flexible middleware capable of integrating complex applications in
heterogeneous environments.
1.9 Thesis Objectives
With regard to the objectives of this Thesis, the target is twofold.
•
An investigation of the application of the CORBA technology has been
carried out in order to tackle the mentioned problems of distributed control
systems. This has been done in the scope of large control systems and in that
of small embedded control systems. Extensions and/or modifications to
existing CORBA specifications for real-time are proposed and implemented
and the validity of the approach demonstrated in numerous examples of
18
•
application (see Chapter 7). For this, a real-time CORBA ORB has been
completely re-built.
A methodology for the construction of distributed control systems based on
the CORBA middleware is introduced. The purpose of the methodology is the
reduction of cost and time in the construction of this type of systems. Of
course, the final objective of the methodology is to ensure that the desired
properties of the system will be achievable by implementation.
The ICa ORB and the ICa methodology (ICm) are cornerstones of the Integrated
Control Architecure (ICa) vision.
1.10 Thesis Structure
The Thesis is organized around seven chapters.
•
•
•
•
•
•
Chapter 1 is the introductory chapter. It explains, as the reader already
knows, the intrinsic properties of the distributed control systems.
Heterogeneity, complexity and the need of integration are among the most
important factors to take into account.
Chapter 2 is an overview of state-of-the-art distributed control systems that
can be found in the marketplace by leading companies. The chapter also
provides a review of the most widely used communication protocols in the
industry.
Chapter 3 presents the different types of middleware technology and the
intended use of each of them. It is frequent to find combinations of
middleware technologies in large systems. MOMs or transaction monitors at
the business layers and other types of middleware technology at lower levels
of control like OPC or CORBA.
Chapter 4 introduces the OMG’s CORBA technology and the basic
specifications relevant for CORBA Control Systems. It addresses issues as
real-time, embedded development, fault-tolerance or specialized services with
real-time requirements.
Chapter 5 introduces the ICa ORB for CORBA Control Systems and the
enhancements of CORBA specifications to deal with issues of real-time in
distributed control systems.
Chapter 6 contains the methodology for CORBA Control Systems
development. The methodology makes considerations about architecture and
19
•
•
modeling and exposes a method to specify requirements, analyze, design,
develop and deploy CORBA Control Systems.
Chapter 7 exposes several examples where the CORBA Control Systems
approach has been used. All are examples of complex distributed control
systems that have been successfully developed and deployed.
Chapter 8 summarizes the achievements of the performed work and develops
some insights on the future trend of research and promotion of the technology.
Finally, two annexes are appended to the document, one introducing a glossary of
important terminology and another one including relevant bibliography.
20
Chapter 2
Industrial Control Systems and Protocols
21
2.1 Distributed Process Control
This chapter deals with the actual state of the art of distributed industrial control
systems. The focus is set on distributed control systems as putting leverage to the
development of such control schemes by means of CORBA is the focus of this Thesis.
The main problem of this type of control keeps being complexity [Åstrom 00]. The
problems of performing process control are combined with those of developing a
system which is software intensive and distributed. This work is not concerned about
the complexity of the control algorithm itself but about the complexity of
implementing system distribution for control purposes.
The second problem in importance is that of integration [Alarcón 94]. It is common
that distributed control systems are huge systems where multiple subsystems coexist.
In fact, defining the boundary of what a subsystem is can be a matter of discussion.
Many subsystems carry out a well-defined purpose independently and can be
considered systems on its own. However, in modern industries, information flows to
and fro subsystems for many different purposes, be them operational (production),
tactical (advanced control) or strategic (administrative). Accordingly, all subsystems
or systems must be integrated so the enterprise can be efficiently run from its helm.
Traditionally, integration has been a source of painful headaches in all types of
software involved businesses. Control has the additional problem that deals with real
world things and therefore can either be dangerous or provoke loss. Integration should
leave no doubts whether reliance can be put on the system or not. ICa solves this
problem by providing a proven framework for integration.
A third problem is that of flexibility [Bengtsson 04]. Large control systems are many
times deployed under ad-hoc designs that impose a rigid structure to the business. This
means that the design tried to solve current problems of the process control but did not
envisioned the future needs of the business; and control does not exist without a
business. Actually, business is under high pressure to adapt and react to changing
market conditions. This implies that companies have to adapt their systems to new
production environments and accept modifications as quickly as possible, usually
before the competition. How the system is adapted, modified, enhanced to
accommodate to market and production demands is another big issue and the root of
enormous expenses. CORBA-like technologies are able to handle this type of
situations.
A fourth issue is standardization. Automation system and device (sensor, IED or PLC)
manufacturers make large efforts in distinguishing the interfaces to their products and
large stakeholders try to impose their “standard” technologies. CORBA [OMG 02]
23
provides a standard specification approach agreed by some 500 companies in the
world. Standardizing interoperability protocols greatly overcomes the limitations of
modifying or enhancing control systems while enormously reducing their cost.
2.1.1
Types of Distributed Systems to Control
It is important to elucidate the background in which ICa has been developed. As with
all tools that are thought with flexibility in mind, there is always a compromise to be
made. The compromise made can be expressed in many forms; e.g. features versus
memory footprint or others. When the work in ICa began, the initial focus was put in
easing the development of large industrial control systems. Examples of such systems
are cement plants or petrochemical refineries. In fact, some of the research projects in
which our group has been involved over the years dealt with distributed control issues
in these types of industry.
After the initial releases of ICa, the idea grew in our minds that the possibilities of the
broker could be expanded to other types of distributed systems. Primarily, embedded
and real-time systems (this including hard real-time systems). There was an initial
attempt to solve problems of large distributed systems where real-time features were
not so important (either there were soft deadlines or the time constants of the
controlled plants were large enough to be obviated). So ICa can be employed in
different versions which interoperate and which are tailored for different systems. ICa
can be used in full, minimum and real-time versions of the ORB. This way it is
possible to use its full potential where there are no resource constraints or to build a
minimal ORB optimized for embedded systems and to run the ORB at the same time
over a hard real-time transport protocol.
Notice that taking ICa to small devices and enabling it with real-time capabilities
allowed to embed the broker in IEDs (Intelligent Electronic Devices) which were the
last barrier in order to achieve full integration from a plant-wide point of view. Thus,
it is possible to establish a CORBA bus that spans the whole industrial organization
horizontally and vertically, from the administrative workstations to the field devices
that run simple control loops closest to the process.
In the following sections, an overview of modern automation systems is presented
along with a variety of communication protocols that are used for industrial purposes.
24
2.1.2
Layers of Control
When complex systems are designed, there is more to it than simply concentrating on
the functional role the system must perform. In the case of complex control, there are
multiple tasks that must be performed which can be logically classified into different
levels or layers. These tasks range from the most basic control loops to the planning
and scheduling of plant operations or production. Instead of building a system in
which all functionalities are included in an unordered way, structure in given to the
system in the form of layers [Gamma 94]. Thus, the control system is logically
composed of layers that form the logical architecture of the control system.
The layered architecture of the system is useful because it reduces complexity as
functional parts of the system can be grouped together and developed independently.
Layers minimize coupling because communication is limited to neighbor layers and at
the same time the layer structure enhances reusability. Layers offer generic services
that are used by upper layers so in the simplest case, each layer only needs the services
of the layer immediately below it. In the case of control systems, lower layers also
need to send information to the higher level layers. This information can be forwarded
in the form of events which are managed by event handlers. At the implementation
level this can be achieved by means of variations of the Observer pattern which
enforces a publish / subscribe mechanism for the events of lower layers. Whether this
mechanism is reliable or not is a matter of design and implementation.
As said, layers of distributed control systems are a consequence of addressing
complexity. Complexity comes also from the different types of information being
handled in the system. Lower layers deal with signals closely related to process
control as sensor related data and set-points whereas in the upper layers more
elaborated information for production planning or emergency management is needed.
This is well known in data warehousing where operational data are clearly differenced
from those that are tactical or strategic.
Regarding the number of layers, there is not a predefined number that fits all systems.
Layers are everywhere. In the smallest distributed systems there are lots of layers if
the system is carefully observed. Take for instance an operating system or a
communications protocol stack where increasingly complex services are built atop
simpler ones (e.g. FTP atop TCP/IP). If focus is put just on the control system itself,
there is a great range in the number of layers from system to system. Small embedded
systems can be run with a single layer as its purpose is to efficiently control a process
probably in hard real-time (e.g. an ABS system) while in the most simple case the
25
pervasive three layer approach can be used (e.g. the planner, executive and functional
layers of NASA’s robots).
It is important to notice that the “universal” three layer approach may not correspond
to system architecture from the control system point of view. The three layer approach
may only fit the system development point of view in which a conceptual separation
between data-source, control logic and presentation layer has been made. But this
approach does not necessarily fit the control need of any individual industry. In fact,
there are different points of view to the system architecture. There is a layered
structure that belongs to the control system architecture (e.g. simple loops, complex
loops, tactical control, strategic control) and another layered structure that guides how
each layer is implemented. This last layered structure is where the pervasive three
layer architecture approach mostly belongs.
So a complex control system architecture style is heterogeneous. That means that
several architectural styles can be employed in a system depending on which part of it
attention is paid [Bass 98]. In the common case, control systems show a layers-oflayers architecture, some are good to handle control complexity while others are good
to control development or implementation complexity. Figure 4 shows a possible
hybrid architecture for a control system.
Control Layers
Perspective
Development Layers
Perspective
Layers
Control System
Hierarchical Heterogeneity
In System Architecture
Figure 4: Hybrid architecture of complex controllers
26
2.1.3
Process Control Elements
There is a large amount of process control elements available in the market today. The
elements to be used in any singular application basically depend on the process to be
controlled. This way industrial control systems are organized into areas of application.
For instance, there are specific industries for chemical process control, power and
utilities, automotive, semiconductor, water, military, aerospace, etc. Although the
basic structure of control is largely similar in all of them, all industries have their own
peculiarities that have given way to specialized providers of control systems.
In the most basic configuration of distributed process control, the main elements are
the sensors, the actuators, the controllers and the communications between them. It is
important to notice communications as a differentiated element of process control as
jitter, latency, transmission delay, queuing delay, etc. in communications pose their
own challenges to process control [Liu 00].
In a real world plant, process control is somewhat more complex that the simple
model of sensor-actuator-controller. This model is useful for basic loops but does not
fit well in the distributed environment of a plant (e.g. a chemical plant) where different
processes are interdependent and so are their process controls. In this type of
environment appears an upper layer of control dedicated to Human Machine Interfaces
(HMI) and Supervisory Control And Data Acquisition.
The HMI mission is to allow plant operators to check process and control elements
status from anywhere in the plant and to help them adjust the control settings for the
process. In doing this, the HMI and SCADA isolates operators from low level details
of control by means of simplified display views of the process where only relevant
data is shown.
But also, in a simple real-world environment, some of the process information must be
shared at the business administration layers and, on the other way, information from
administration must go down to the process (e.g. to adapt production to client orders).
This means that the process control must also be integrated or must interface with
Manufacturing Execution Systems or MES. MES are software systems that track and
manage all aspects of a job in real-time while it is in process (a.k.a. execution phase).
27
While MES takes care of resource scheduling, capacity planning, process management
or data collection from computerized instruments in the short term there are also
control needs that must be addressed in the medium or long term. This is the mission
of Enterprise Resource Planning (ERP). An ERP incorporates to the industry a higher
layer of control which is specialized in managing product planning, parts purchasing,
customer service, inventory maintenance, supplier interfacing, order tracking, etc.
ERPs achieve all these tasks by the integration of multiple software modules.
As can be easily understood, the landscape of complex process control is very
involved. If this is combined with the lack of standard ways of integration there is a lot
of effort in labor and money that goes into eliminating the friction between the
modules that compose the distributed control system. This is precisely one of ICa’s
goals, to provide out-of-the-box integration at all levels of the control hierarchy.
2.1.4
Sensors, Controllers, Actuators
At the same time that control tends to become more complex, the computing power is
becoming cheaper and cheaper. This situation is creating favorable conditions for the
introduction of intelligence (controllers) close to the process, at the sensor and
controller levels. Intelligent Electronic Devices (IED) are common instruments in all
industries nowadays.
One of the best examples of the use of IEDs is in the power substation automation
industry. Thanks to IEDs, complex tasks in substation automation can be performed.
In today’s substations, IEDs for protection, metering, data gathering or control
purposes are a reality.
Figure 5 shows the GE D25 multifunction IED [GE 03]. This IED has been
specifically engineered by GE for power substation automation. The D25
multifunction IED for substation control includes a Definite Time Over Current
Protection algorithm, and breaker failure protection feature. The product supports
synchronizing functions, monitors voltage difference, phase angle difference, and slip
frequency and supports multiple buses per feeder 1 . The unit also functions as
programmable logic controller, substation local area network node, and IED gateway.
The advantage of this is that the unit can replace several devices with a single unit.
With the use of IEDs for electricity transmission and distribution control, substation
automation projects can be managed more efficiently as the number of elements
1
This is electrical systems terminology.
28
needed to monitor the plant decreases. It must be noticed that although the complexity
of the automation project decreases, the complexity of the control elements (e.g. the
IEDs) has increased. For this reason, modern IEDs incorporate self-checking functions
in order to verify system status.
While the number of elements in the plant decreases with the use of IEDs, the
software needed to manipulate the plant is more complex as the number of services
provided by IEDs has grown. The advantage is that better control of the plant can be
exercised as there is more information to reason upon.
Figure 5: A multifunction IED from GE
But among the objectives of this Thesis is not only to ease the work of developing
distributed control applications for large industrial complexes but also in the area of
distributed real-time embedded (DRE) systems. The focus also includes hard real-time
applications. An example of this type of systems is the real-time control of automotive
systems. In this case, independent autonomous subsystems must share information in
hard real-time. For instance, the engine and the transmission subsystems of an
automobile must share information in such a way that there is a maximum bounded
latency between the input stimulus and the response. The type of sensors used in this
case has stricter requirements that those commonly found in large plants. Also the
resources available in the sensors, controllers or actuators are smaller (e.g. computing
power, available storage space, memory footprint, etc.).
Figure 6 shows a drive-by-wire box [TTTech 05] which supports communications
based on the Time Triggered Protocol (TTP) [TTP 02]. This box is designed to be
29
used in automotive hard real-time applications where high power actuators are needed.
The box is suitable for brake-by-wire, clutch-by-wire or steer-by-wire applications.
The TTP box is also an actuator that supports direct control of a brushless DC motor.
Figure 7 shows a high-performance snow groomer. This machine electronic core is
based on a TTP network which controls the driving systems and the overhead winch
so engine power can be easily adapted to operating conditions. The rear snow tiller,
the front rake blade and the entire on-board instrumentation are deployed on the TTP
network.
Figure 6: A TTP drive-by-wire box from
TTTech
Figure 7: A snow groomer based on a TTP network (figure authoring
of TTTech)
In summary, the availability of better hardware resources even for embedded real-time
applications has opened the path to the integration in a variety of devices of software
architectures for control purposes that were unthinkable a few years ago when most
control software developments were ad-hoc for every system. For instance, the use of
30
software frameworks was impeded by the scarce computing resources of deeply
embedded systems.
2.1.5
Industrial Communications Protocols
When talking about communications for industrial applications [Hanssen 03] there are
several levels that must be addressed. Here, in this first part of the section, referral is
made to large plants (e.g. petrochemical plants) and not to embedded applications.
Those levels approximately correspond to the different types of networks that can be
found in large distributed industrial applications. These networks are frequently
denominated field bus, control and enterprise (also business or information) networks.
Going bottom to top the first network is the field bus network.
The field bus network mission is to link the distributed intelligence of IEDs so they
can meet their tasks. Control is decentralized as much as possible so each device is
able to gather the needed data from the field bus to carry out its work with local
intelligence being decoupled as much as possible from a central control (control
network). Field bus technology has evolved much in recent years. The problem with
field bus technology is that there is not a unifying standard of field bus technology.
Different field buses coexist in the market (e.g. Profibus PA [Profibus], DeviceNet
[Dev], World FIP [World]). Differentiation came in one way because of specialized
needs of industries like automotive, pulp and paper, process or semiconductor.
Secondly, differentiation of field bus technology came because of the race for the field
bus market among Japanese, European and U.S.A. companies. This situation has lead
to the development of several field bus communication standards each of which is
employed depending on the industry, geographical area (see Figure 8) or control
system implementer’s preferences.
Figure 8: Geographical distribution of field bus usage
31
Field bus networks are usually small, exchange low amount of data and have typical
reaction times of about 10 ms.
The second network is the control network. As happens with the field bus network
there are several protocols available. Most of them are based on Ethernet. Examples of
these protocols are ControlNet [ControlNet], Profinet [Profibus], Foundation Fieldbus
HSE [Foundation], Modbus [Modbus] or CAN [CAN]. The requirements for this
network are less stringent than those of the field bus network. They are medium sized
networks which support high amount of data and typical reaction times of about
100ms.
The enterprise or information network is a typical TCP/IP network. Its size can be
very large, very large amounts of information are exchanged and the typical reaction
time expected from such a network is about 1 s. An example of the control network
layers for a continuous process plant is shown in Figure 9.
ENTERPRISE NETWORK
MIS
Safety
Business Management
Data Storage
CONTROL NETWORK
Process Control
Process Operation
Process Management
FIELDBUS NETWORK
Field Configuration
Sensing and Acting
Field Management
Figure 9: Process plant integration
32
Many of the communications protocols described above are not suitable for hard realtime. When critical systems come into play a more deterministic approach to
communications is needed. Among the systems of this category we can cite railway
signaling systems, fly-by-wire cockpits or automobile computer networks. Examples
of more deterministic communication protocols are RT-Ethernet [Doyle 04], the Time
Triggered Protocol (TTP) [TTP 02], FlexRay [BBE 02], and the time-predictable
variant of Control Area Network (CAN) [Bosch 91], TTCAN [Har 00].
The TTP protocol approach relies on system progress based on time rather than on a
system model that progresses by events. The idea is that event-triggered systems
cannot easily cope with the requirements of composability, predictability, determinism
and guaranteed deadlines of hard real-time systems. TTP relies in a Time Division
Multiple Access (TDMA) scheme to control access to the communication medium
(see Figure 10). In this protocol, access to the bus is controlled by the progression of
time where the available channel bandwidth is statically divided into a number of slots
which are used by the network nodes for their transmissions. The TTP uses a priori
knowledge to schedule the network. This means that the access to the communication
medium by each node is known at design time.
Figure 10: TDMA bus access scheme in TTP
TTP has several flavors; mainly TTP/A and TTP/C. TTP/A is member of the TTP
family intended for low cost networks of sensors and actuators while preserving the
temporal properties of TTP for field bus applications. The TTP/C protocol of the
family is intended as a high speed network (25 Mbit/s) for high dependability
applications. TTP/C provides fault-tolerance by the ability of tolerating one arbitrary
faulty node or one faulty communications channel in the system. TTP/C can be
integrated with TTP/A in an architecture where several field bus clusters have to be
interconnected in a temporally predictable manner.
FlexRay is based on the TTP/C protocol fundamental ideas. It is specifically designed
to be introduced in the car due to the amount of electronics for automotive
applications of recent years. The FlexRay protocol provides fault-tolerant time
33
synchronization via a global time base, collision-free bus access, guaranteed latency
and system fault-tolerance by the use of dual communication channels.
The Control Area Network (CAN) is a serial protocol that supports the prioritization
of messages. CAN supports data rates up to 1 MBit. The medium access scheme used
by CAN is a Carrier Sense Multiple Access / Collision Avoidance (CSMA/CA). The
problem with CAN is that worst case latency can be too high. To make CAN latency
predictable TTCAN can be built on top of the standard CAN protocol. TTCAN
synchronizes the schedules of all the CAN nodes in the network by means of a global
system time base.
Among the several candidates for a real-time communication transport protocol for
CORBA, a decision was made in favor of a RT-Ethernet plug-in, developed in
collaboration with the Department of Automatic Control at Lund University (Sweden)
and a TTP/C based transport plug-in that was developed for ICa in collaboration with
the Institut für Technische Informatik (ITI) at the Technical University of Vienna
(Austria) which is directed by professor Hermann Kopetz, chief architect of the Time
Triggered Protocol. The TTP technology was selected because of its increasing
relevance in railway, aerospace or automotive markets. For instance, Honeywell uses
TTP for the digital engine control system of Lockheed Martin’s F16 fighter aircraft.
Another example from the aerospace industry comes from Nord-Micro, which uses
TTP as the communications protocol for the Airbus 380 cabin pressure control system.
RT-Ethernet is a cheap approach to real-time applications that gives good performance
for many (non safety-critical) systems on standard communications hardware.
2.1.6
Trends of Control Networks
Although in the process industry always there have been the control levels of Figure 9
(with their differentiated enterprise, control and fieldbus networks), the control
implementation scheme for large and small industries keeps evolving over time.
Control system construction and deployment has evolved from the direct digital
control, where all devices were connected separately to the control room (where the
control was centralized to a single computer), to the traditional Distributed Control
System implementation, where several devices are linked to a controller and there are
several distributed controllers that are connected to the DCS console. In the years to
come, control will be totally distributed to field devices which integrate all needed
control loops. This will be the era of true distributed intelligence. Figure 11 shows
such evolution of complex control system implementation where the decentralization
process is shown against time.
34
PAST
(Centralized)
Centralized Control
Distributed Control System
PRESENT
(DCS)
FUTURE
(Network Integration)
Distributed Intelligence
Figure 11: Trends of control system networks
2.2 Commercial State Of The Art DCSs
The purpose of this section is to present an overview of commercial distributed control
systems that represent the state of the art of technology.
2.2.1
Honeywell Experion PKS
Experion Process Knowledge System [Honeywell 04] is a system solution from
Honeywell which integrates all networks / layers of Figure 9 under a unified system
from the same vendor. Experion is a single solution for the whole plant. Experion goes
further than the process control a simple DCS can afford by including subsystems to
35
manage the plant at the enterprise or business level. Figure 12 shows the architecture
of Experion PKS.
Figure 12: Platform architecture of Experion PKS (figure authoring
belongs to Honeywell)
As can be seen in Figure 12, the architecture is organized in layers where four
different networks are differentiated. First, there is a Local Control Network which is
the fieldbus network. The process network is divided in Experion in two networks, the
Supervisory Control Network and the Advanced Control Network. Finally, the
enterprise or Business Network is placed atop all other layers.
Experion improves former Honeywell solutions with improved process applications,
plant asset management and integration of business services. For instance, Experion
asset manager monitors the plant in real-time and helps to discover areas that are most
likely to have a negative impact in performance or determines how well a device is
performing in the context of a control loop. This kind of monitoring is useful to
minimize downtime and to avoid unscheduled replacement costs.
36
Another problem in large complex plants is the operator decision system. Knowledge
related to process improvement or plant safety is frequently concentrated in
experienced individuals and is rarely captured and shared. In practice, this means that
non-automated tasks are treated differently by different operators resulting in a loss of
system overall performance. Experion PKS includes collaborative decision-support
tools to let operators to prevent abnormal situations while maximizing plant
production.
From the business point of view, the Enterprise Network, Experion offers tools to
analyze production data in real-time so performance reports can be generated
continuously and not on a quarterly basis. Experion offers interfaces to pure business
systems so the complete plant can be integrated from planning and scheduling to the
supply chain.
One of the problems of current systems is integration; Experion tries to be open by
providing integration services with other non-Honeywell systems. All this integration
effort always has a cost which is finally satisfied by system buyers. For instance,
Experion provides support for the following field buses: FOUNDATION, HART,
Profibus, DeviceNet, LON, ControlNet and Interbus. The question remains how much
does this complexity for multiple communication mechanisms add to system cost.
2.2.2
Airbus 380 Cabin Pressure Control System
The Airbus 380 cabin pressure control system is another example of a distributed
control system that can be tackled with the ICa ORB. The Airbus 380 is the largest
and most advanced airliner to date. It is a double-decker with space for 555 seats in
which the seat × mile operating costs have been reduced 15-20%. The control of cabin
pressure and ventilation system in such a large aircraft is somewhat complicated.
First, it must be understood that the cabin pressure control system is safety critical.
Additionally, the pressure in the fuselage must be comfortable for the passengers and
the crew in any altitude. At the same time, it is necessary to reduce complexity in the
control system and to make it as lightweight as possible. These two conditions mean
that it is necessary to develop a distributed real-time system with safety critical
requirements and that wiring should be as simple as possible. The wiring problem can
be solved by means of a communications bus. The real-time and safety-critical
problem (from the point of view of communications) can be solved by means of a
suitable protocol and hardware. In the case of Airbus 380 this protocol is TTP/C due
to safety requirements.
37
The control and monitoring system of cabin pressure operates autonomously
controlling the air volume that flows out of the fuselage through four outflow valves.
The system incorporates two safety valves and a negative-pressure relieve valve for
safety. Figure 13 shows a schema of the control system with duplicated TTP
communication channels for fault tolerance.
Figure 13: Airbus 380 cabin pressure control system schema
The control loop for this system is quite simple. The pressure controllers input
variables are the cabin pressure, the outside pressure and the flight profile data
(altitude and speed are important for pressure prediction). The controller generates a
control signal for the required actuator to set the outflow valve angle. All
communications take place by means of TTP messages.
2.2.3
AVE Power Distributed Control System
AVE stands for “Alta Velocidad Española”, the high-speed railway system of Spain.
The AVE railways are powered by electricity distribution substations geographically
scattered along the railway path. The distributed control system of the power
substations is interesting because it combines real-time, non real-time and fault
tolerance requirements. This distributed control system and the power substations
38
have been implanted in Spain by the consortium formed by the Spanish company
Electren and the French company ALSTOM (the AELEC consortium). Figure 14
shows a schema of the control system architecture.
The main idea behind the architecture is that the power system for the railway is
divided by traction substations that have several associated auto-transformation
centers. In the figure, the traction substation is depicted in the upper left corner of the
image inside the dotted-line square. The bottom half of the picture shows the autotransformation centers associated to the traction substation. In the upper right corner
there is a remote central telecontrol post. In each of the traction and autotransformation centers there are several networks. First there is a primary network
which is represented by the double rings in each of the centers and that links the center
PLCs (in black). The secondary network links the PLCs of each ring with their
controlled devices (the IEDs, in blue, which control circuit breakers, switches, etc. and
that are not shown in the figure for simplicity purposes).
Figure 14: AVE power substations distributed control system (Figure
authoring belongs to AELEC)
39
In this case, those devices are mainly protection relays (fully fledged IEDs with
complex functionality) which are depicted in blue color. There is a third network
which links all traction and auto-transformation centers along the railway line. Fiber
optics is used extensively for all networks except for some protection devices which
are tripped by means of copper wire links.
The primary network is based on the EFI.P protocol and is physically redundant. The
protocol is deterministic and the network links all PLCs and a redundant industrial PC
used as a data server and gateway to the external (the third) Ethernet network. This PC
also serves as a time base for all events in the substation network by means of a GPS
card.
The secondary network uses two types of communication protocols with the
protection relays: MODBUS and IEC-870-5-103. The first is a fieldbus protocol for
serial communications. The second is a specific protocol for communication between
Remote Terminal Units (PLCs) and protection devices.
The third network uses Ethernet and is the operator applications network. All control
applications on this network use OLE for Process Control (OPC) to communicate with
the data server which acts as a gateway between the second and third level networks.
Finally, there is a fourth network that uses the protocol IEC-803-5-101 to
communicate with the remote telecontrol post. There is an OPC / IEC-803-5-101
gateway to provide the conversion between protocols (middle of the upper half in
Figure 14).
The SCADA of this control system is based on a General Electric SCADA
development tool called CIMPLICITY and its runtime engine CIMVIEW. It also uses
at least six different communication protocols which make integration of the different
parts of the system complicated and, consequently, raise costs and risks. Among the
integration problems posed by this system there are some interlocking operations
which must coordinate elements of the traction and auto-transformation substations.
For this, there are some automatisms running in a control PC connected to the
Ethernet / OPC network running on the ISAGRAPH software.
This system structure corresponds to the present state of distributed control systems of
Figure 11 as is the state-of-the-art solution of section 2.2.1. The difference between
the two is that the AVE system is a real example of the integration of hardware and
software from multiple vendors and the problems (and advantages such as avoidance
of vendor lock-in) this implies.
40
2.3 Critique Statement
The three reviewed systems show some examples of current state-of-the-art distributed
control systems. First, a whole solution from a world-class manufacturer is shown.
This solution addresses process control of large plants where the promise of seamless
integration of the hierarchy of control layers is delivered. The main problem with this
solution is that vendor lock-in is very difficult to avoid. In some cases vendor lock-in
does not need to be bad but, common sense indicates it is better to have a wide variety
of suppliers to buy products from. Vendor lock-in eliminates competitors and that
almost always results in an imperfect market. Although this kind of systems integrates
with other systems and protocols, it always tries to promote their own, so the benefit
for the DCS manufacturer can be maximized. The problem here is that proprietary
technology is imposed to system proprietors. Interoperability must be independent of
DCS vendors. ICa can help solve this problem by means of the public CORBA
specification of interoperability.
The second system shown, the Airbus 380, is completely different. It is a distributed
system with hard real-time, fault tolerance and safety-critical requirements that is
developed mostly ad-hoc. In this case, the choice of a suitable communications
protocol is less wide because of the stringent system requirements. However, the
communication system to be employed (TTP/C) lacks the tools to ease the
development which results in higher costs in an area of control in which development
costs already result very expensive. Also, there are not suitable frameworks for the
development of real-time applications. For instance, tools to control processor,
memory or communications resources. ICa assists in solving this problem by
providing this kind of control.
The third system, the AVE power substation control system, is a large system
geographically scattered which takes the multi-vendor integration approach and in
which different types of requirements for the different types of networks in the system
must be met (from real-time to non real-time but always fault-tolerant). In this case the
problem is twofold. First, there is the integration issue of products from multiple
vendors. There is no consensus on hardware platforms, operating systems or
communication protocols. Second, there are different types of requirements for the
different parts of the system. ICa solves these problems by a) adopting an
interoperability protocol that can be adopted by all manufacturers, b) providing ORB
releases with real-time, non real-time and embedded characteristics.
41
Chapter 3
Distributed Component Middleware
43
3.1 Definition of Middleware
Middleware can be accounted for as the glue that links together all the elements of a
distributed system. It is the piece of software that mediates between applications and
the network and that isolates the application from many of the lower level details of
the operating system. Middleware aids in carrying out the interaction between
components across a heterogeneous distributed system. There are several types of
middleware (see Figure 15) but the one with most wide acceptance is the Object
Request Broker (ORB) middleware. This is the result of its object orientation,
providing the advantages offered by objects to the distributed world.
The objective of middleware is to release developers from the common, complex, lowlevel or mundane tasks of module interaction in a distributed system. This is achieved
by implementing a common abstraction for the programming of distributed systems.
All modules of the distributed system must conform to the abstraction in order to be
able communicate. Thus, middleware is used to make processes residing in one or
more host computers to interact. The middleware provides to applications a set of
services of coarser granularity than those of the operating system in the form of
Application Programming Interfaces (API) that enable them with the following
properties among others.
•
•
•
•
•
To locate services across the network in a transparent manner. This means that
the distribution is transparent for applications. For the component parts of the
systems it is as if the system were monolithic.
To be independent from network services. The applications are not aware of
the network; the middleware is in charge of managing the intricacies of the
network access details.
To be reliable and available. Programming the network is a complex task and
the middleware provides a proven and reliable way to glue together the
components of a distributed application.
To scale up in capacity without losing function. Middleware provides services
to control the lifecycle of the components of the distributed system and to
control the resources available for the whole system.
To resolve the issue of heterogeneity, allowing the construction of systems
where there is heterogeneity of hardware, operating systems, network
protocols and programming languages.
45
Application/
Industry/Business
aware
Applications
Vertical Domain Services and Frameworks
(Telecomm, Groupware, Utilities, etc.)
Distributed Object Services
(CORBA, DCOM, ActiveX)
Client / Server Services
(RPC, MOM, TPM)
Primitive Services
(Terminal emulation,
File transfer, email)
Network Programming Services
(Sockets, LU6.2)
System
aware
Network Services
(TCP/IP, SPX/IPX, ATM,SNA)
Figure 15: Types of middleware services
3.2 Taxonomy of Middleware
Middleware can be classified according to the four following forms:
•
•
•
•
Transaction Processing Monitor Middleware
Remote Procedure Call Middleware
Message-Oriented Middleware
Object-Oriented Middleware.
The classification distinguishes the different types of middleware according solely to
the different types of programming machinery involved in the communication
between parts of the distributed system. Essentially, the criterion is based in that all
types of middleware define a uniform scheme to access other components of the
distributed system scattered across the network.
3.2.1
Transaction-Processing Monitor (TPM) Middleware
TPM is a client / server middleware specifically designed for transaction applications.
This is a mature software technology that is employed in applications involving data
management, database access, order delivery, etc. The TPM middleware is designed to
provide access to thousands of clients in a distributed client / server environment by
46
controlling and mapping client transaction requests onto a set of service routines that
multiplex the transactions (see Figure 16).
The service routines are critical to the architecture of the system and are the only
client-routines that the server system interacts with, avoiding the overhead of dealing
with thousands of clients. The advantage of this technology is that the server system
only sees a reduced number of interactions through a small set of routines and not the
whole amount of clients. Client access is prioritized according to the type of service
requested. Also, the logic from the business rules of transaction processing can usually
be moved from the clients to the transaction processing monitor making it easier for
administrators to deploy system upgrades.
TPM technology scales well when the number of clients grows as all requests are
multiplexed by the TPM. When the number of clients is too large, it is possible to add
more TPMs to better handle multiplexing of requests. The TPM market is dominated
by a few vendors. They provide large systems that integrate with a great variety of
business processes and offer ways to access the functionality of the TPM in several
ways. For instance, Tuxedo TPM from BEA has a CORBA and an Application-ToTransaction Monitor Interface (ATMI) interface to access the TPM infrastructure.
client
client
client
client
Service
Routines
client
Transaction
Processing
Monitor
client
Server
client
client
client
client
client
Figure 16: Transaction Processing Monitor
Most of these TPM products offer advanced features as load balancing, management
features or fault tolerance features. Some of the most widely employed TPMs are:
47
• IBM CICS: CICS stands for Customer Information Control System. The
name stands for a family of application servers and on-line transaction
management infrastructure for mission-critical applications. CICS has
interfaces available for COBOL, C, C++, Java or PL/I.
• BEA Tuxedo: The product includes an application server, a TPM, a backend transaction processing infrastructure and two application programming
interfaces one for CORBA in the C++ language and the ATMI.
• HP Encina: Encina uses the Distributed Computing Environment as an RPC
basic infrastructure over which the transaction processing monitor is built. HP
has enhanced the RPC mechanism to give it transactional semantics defining
what they call Transactional RPC.
3.2.2
Remote Procedure Call (RPC)
The RPC is a client / server infrastructure that allows developers to access components
of a distributed system by means of function calls. In the case of RPC there is not a
defined software layer that implements the middleware, middleware is embedded in
the client and server applications.
To embed the Remote Procedure Calls, at compile time stubs are created for the client
and server sides which are compiled and linked with the rest of the application. The
stubs are invoked when the client program calls a function which must be serviced by
the remote RPC server program. This mechanism is depicted in Figure 17. RPCs are
synchronous calls between clients and servers although some vendors offer
asynchronous functionality. When a remote function is invoked, the calling procedure
(the client) must wait for the server to end the processing of the function before
control is returned to the client application. Notice that the function call blocks until
the server application replies. This mechanism can be chained if the server is at the
same time client of another application and invokes another remote server when
servicing the original RPC.
The advantage of the RPC mechanism is that it masks the function calls as if they
were being made locally although the inner service workings behind the masked
function call are rather complex. Function arguments must be marshaled and a
message to the remote RPC service where the function is implemented must be sent
and the reply returned. This is generally done synchronously so a tight coupling
between the client and server programs exists, being it possible for a client to remain
blocked for a long time in a lengthy procedure call. To keep the system running with
this kind of infrastructure the only solution is concurrent programming, multithreading
the system which in turn increases complexity.
48
Server
Transport
Network
Transport
Client
RPC
Stub
Network
The major drawback of RPC is that it is a relatively low level mechanism. The details
of the underlying communications mechanism are very little hidden from the
developer. RPCs provide support for a procedural approach to system development
and, while in some cases this might be good there are myriads of situations in which
an object or component-based approach is significantly better. Being the
communication model synchronous increases the complexity of development in the
case lengthy remote procedure calls must be done.
RPC
Stub
Application specific
RPC
Figure 17: Remote Procedure Call Middleware
There is a variety of vendors in the market which provide RPC software with both
synchronous and asynchronous functionality, among them there are the following
products:
•
•
•
Entegrity Solutions Gradient RPC. Entegrity is a provider of business to
business solutions based on the DCE RPC. Gradient RPC difference from
previous RPC products is that the communications employed for RPC are
secure as demanded by business to business communications.
Compaq’s (HP) DCE. Compaq sells RPC products based on the Open
Group’s DCE for the OpenVMS and True64 UNIX operating systems.
Sun’s RPC. As the company is nowadays oriented towards the Java language,
it currently offers RPC products which expose a Java API for XML-based
RPCs (JAX-RPC). XML-based RPC are remote procedure calls specifically
designed to be invoked over the internet. For this reason, the transport
protocol used is HTTP and the encoding is XML. Hence its name, XML-RPC.
The purpose of this product is the interoperability of web services among
different platforms and languages.
49
3.2.3
Message Oriented Middleware (MOM)
The MOM is a client / server architecture that supports asynchronous invocations
between clients and servers. In a MOM, software architecture messages are not
directly sent to the server, there is a message queue at the client and server sides (see
Figure 18). The purpose of the queue is to store messages in case the receiving
application or process is busy performing another task. Once the sender has released
the message, it can continue its processing. The queue that stores the message will try
to send it to the recipient process until the message is accepted. Once a reply is built
by the recipient, it will be sent to the queue which will try to send it to the original
sender of the message until it is again accepted.
The advantage of MOMs resides in that it decouples clients from servers by means of
the queues. This architecture allows MOM-based applications to process huge
amounts of messages as they are stored in the queues and that will be processed when
there are computing resources available. Most MOMs will accept that more than one
process reads messages from a queue, being the system able to adapt message
processing to the number of messages held in the queue. Also, most implementations
support synchronous message passing.
MOMs are well suited for object-oriented systems and for event-driven applications.
For object-oriented systems, the message passing implemented by MOMs fits well
into the peer-to-peer communications model of objects. In the case of event-driven
systems, messages can be passed to the MOM infrastructure when the event occurs.
The responsibility of forwarding the message to the server is then on the middleware
and the client can continue its processing. As the basic communication process is non
blocking (messages are stored in the middleware), there is a potential danger of
overloading the network by issuing thousands of messages to a server which is
blocked or simply that can not follow the message arrival rate. This situation is
corrected in most MOMs by defaulting to synchronous message processing when such
a situation is detected. Unlike, RPCs which always block (although, as mentioned,
there are vendors that provide asynchronous implementations), MOMs provide a more
flexible approach to systems development. Notice, that MOMs are especially good for
applications whose modules cooperate with a certain degree of independence and
where the server may be unavailable at the moment the message was issued. For this
reason, it is well appreciated technology for in Enterprise Application Integration
(EAI).
50
Server
MOM
With message queue
Transport
Network
Network
Transport
MOM
With message queue
Client
Application specific
proprietary messages
Figure 18: MOM Architecture
Among the vendors that dominate the market there are the following:
•
•
•
3.2.4
IBM MQSeries: This MOM has been designed to integrate heterogeneous
environments and supports more than 35 different platforms. The MOM
offers an API called Message Queue Interface (MQI) which must be used
by all applications that want to communicate with other systems. The
MQSeries supports standards such as XML or the Java Messaging Service
(JMS).
Software AG EntireX: EntireX is a component-based application
integration product based on a message server brokering software. This
MOM handles security information and supports synchronous and
asynchronous communication models.
TIBCO Enterprise Message Service, SmartSockets and Rendezvous:
TIBCO manufactures software for messaging and multicast products.
EMS is a message queuing product that supports the asynchronous
communication model, message persistence and transactional
communications. SmartSockets is a product that supports a publish /
subscribe model in which messages are delivered only to interested
applications. There is a version of the product SSLSmartSockets that
provides secure connections among applications. Rendezvous is designed
to increase the scalability for the overall enterprise infrastructure by
providing message routing and filtering services.
Object-Oriented and Component Middleware
There is a distinction between component and object-oriented middleware and the
reminder types of middleware. Object and component middleware key-concept is that
51
of the interface. Every service in this middleware is accessed through interfaces and
no implementation details of the service need to be known by the requesting
applications. However, this is also a concept that is established by the very object
concept. There are some other reasons that back the promotion of this type of
middleware:
•
•
•
Most programming languages do not support distributed system construction,
The object or component model alone does not support distribution across
machine boundaries,
Large systems are constructed in multiple programming languages, operating
systems and hardware. They are based upon heterogeneous environments.
The reasons above are what lead to what are called “stovepipe” systems. Systems
which are developed ad-hoc, with proprietary integration solutions. Instead of this, the
object-oriented middleware defines the requirements to build systems where there is a
standard way for internetworking and where there are standard interfaces to access
common services of the infrastructure. In the object-oriented middleware, there is
support for the object-oriented programming model and thus the same benefits that
object systems have can be applied to this type of middleware. The most important
one is that object systems map well the entities from the real world. Then objectoriented middleware maps well the entities of the real-world that are distributed in
nature. Figure 19 shows a general architecture for object-oriented middleware.
local
object A
proxy
object B
OOM
object
request
broker
/
object
manager
OOM
object
request
broker
/
object
manager
remote
skeleton
object B
object B
Figure 19: General architecture for object-oriented
middleware
Object-oriented middleware uses an Object Oriented Manager (OOM) or an Object
Request Broker (ORB) as the mediator software layer between local and remote
objects. Remote objects expose their interfaces which can be accessed as if they were
local objects. For this, the remote object interface is accessed locally via a proxy
52
object. The proxy object acts on behalf of the remote object and its responsibility is to
forward the request invocation to the middleware layer, the Object Request Broker
that can be thought of as a software bus, which will take the necessary actions to
correctly deliver the request. The basic mechanism used for communication is that of
remote method invocation. Remote objects can be uniquely identified by means of
their object references. Object-oriented middleware has several differences with other
types of middleware:
•
•
•
•
•
•
•
There are neither clients nor servers. In object-oriented middleware there
are only objects that provide services and that receive services from other
objects.
Location transparency. The use of proxies decouples the service user from
the service provider. When implemented, this yields the advantage that the
location of the service can be done independently of the service user.
Design of service distribution can be postponed. Services are provided by
objects and as location transparency is one of the middleware properties, users
of services are unaware of where the computing resources for the service are
located. The system can be developed locally and distribution can be delayed
until system testing. The system is perceived as a whole by the objects, as if it
were a monolithic system.
Migration of objects. Objects in the system can be moved dynamically
without affecting the users of the services they provide. This is achieved by
common services built in the middleware layer. For instance, in the case of
CORBA the ORB is able to issue “locate forward” messages to resolve a new
object reference in case an object is not found in its previous location.
Concept of services. Object-oriented middleware introduces the concept of
common services. These are services that can be provided by distributed
objects in a standard way (e.g. vendors can bundle the services with their
products). Examples of these services are the naming service, event service,
object lifecycle service, etc.
The system architecture is open and scalable. Distributed object systems
are able to scale without modifying the underlying system structure or
architecture. The system simply grows up by increasing its number of objects.
Independence of programming languages and hardware platforms. The
key concept is interface. Distributed Object Middleware uses Interface
Definition Language (IDL) to define object interfaces. The interface concept
is useful because it allows defining a common object model that is expressed
in IDL. IDL is independent of programming languages or implementation
details. At the programming level, IDL is mapped to any number of
programming languages by IDL compilers making the system independent of
53
languages or particular hardware implementations. The specific language code
generated from IDL files contains the proxy code to be used locally to access
remote objects (stubs) and the templates of remote object services to be
implemented (skeletons).
There are three different approaches to object-oriented middleware that dominate the
market, OMG’s CORBA, Sun’s Java/RMI and Microsoft’s DCOM. Each of them has
some advantages and disadvantages that will be presented in the following sections.
3.3 Components vs. Distributed Objects
There is not a clean distinction between components, in a general sense, and
distributed objects. In its definition a component shares two basic properties of
objects; it has interfaces and provides data-hiding. A component is a group of
functions and related data that can be encapsulated into one or more objects that form
the component. The functions can be accessed by a running program residing in the
same or in another host. For instance, a cryptographic component groups functions to
provide different cryptographic algorithms to its clients. The component can hold
several objects, each one being responsible of a different cryptographic algorithm. In
some cases, the component will have only an object, making then difficult the
distinction between distributed object and component. In the common case, the
component is constituted by a set of collaborating objects. In the case of legacy
systems, not an unusual case, a component can be formed by a module of a system
that has been wrapped with CORBA interfaces. Thus, for the scope of this work, the
description of component accepted agrees with that of [Brown 97], “A component is
an independent set of deliverable services”.
A fundamental difference between objects and components is that in the essence of
objects is the representation of some mental or real-world form of the envisaged
system. In the essence of components this idea does not exist. The idea of
componentry is to build software out of components. Software must be componentized
so it can be built from components. Software components are a way to encapsulate
functionality, often in the form of objects, upon a software infrastructure so the
component can live in isolation and autonomously from the rest of the system.
Software components have been built upon the ideas of software objects, architecture,
frameworks and patterns.
It is useful to think of a component as a system module which is linked to middleware.
The transformation of modules into components of a middleware system reduces
54
overall complexity as the modules are easily reachable from anywhere in the system
without making them tightly coupled. Different components of the systems can be
changed or modified independently of the others. Components can be classified
according to many different criteria (e.g. according to component stakeholders,
developers, integrators, clients). In order to stress the difference between objects and
components Figure 20 shows a classification of components and middleware
according to their granularity and to their scope.
Advanced Component
Models
Basic Component
Models
Infrastructure
Components
OLE
CORBA
JavaBeans
Business
Components
Domain
Components
GUI
Component
GUI
Component
Business
Component
Process
Component
COM+
Enterprise
JavaBeans
CORBA
Component
Model
Utility
Component
(Database,
Transaction)
Business
Component
Utility
Component
Smaller, less complex
components
Business
Component
Larger, more complex
components
Figure 20: Classification and types of components
3.4 Open
Group’s
Environment
Distributed
Computing
The Open Group is an international vendor and technology-neutral consortium. It was
formed by merging the Open Software Foundation (OSF) and X/Open Ltd. Its mission
is to help developers, vendors and clients accelerate the development of open systems.
The consortium now has more than 300 members that along with customers unify
their vision and improve consensus-building on how open systems should be
implemented.
The Distributed Computing Environment (DCE) is a piece of software technology to
build distributed systems in heterogeneous environments. DCE was historically the
55
first multi-vendor middleware standard technology. Over the years, DCE has become
a basic technology upon which other middleware for distributed systems has been
built (e.g. transaction monitors). DCE comprises a set of software services residing on
top of the operating system. It is middleware that uses low-level operating system and
network primitives to carry out its tasks. As it was promoted by the Open Group, it is
vendor neutral and has been designed for and runs on multiple platforms. The Open
Group provides DCE in source code form to be bundled by vendors with their
products. DCE usually ships with the operating systems and is available almost on
every platform. The DCE services include,
•
•
•
•
•
•
Remote Procedure Call (RPC). For client-server communication, so
distributed resources can be accessed across a network.
Security Service. This service provides user identity authentication,
authorization to distributed resource access by users and account management
for users and servers.
Directory Service. The directory service provides a naming model across the
distributed system.
Time Service. To provide network-wide clock synchronization.
Threads Service. The threads service is designed to help servers accept
concurrent requests increasing the execution capability of the system.
Distributed File Service. Provides access to files across a network. When
linked to a directory service, common names can be used in the whole system
to address networked resources.
DCE was not thought of as an object-oriented technology. As the use of objects is
increasing in all types of software, there is a significant challenge in what will the
evolution of DCE be. DCE was initially written in C language (not an OO language)
and its only standard specification is written then in that language. There was a first
attempt, with the release of the DCE 1.2.1 standard in 1996, to specify an interface for
C++ but, in practice, the support from vendors for the DCE C++ interface has been
partial and fragmented. This lack of support severely hampers object-oriented
development using DCE. The evolution of DCE must go through the world of objects
where it will encounter severe competition by CORBA and DCOM.
DCE is now a mature product and although it is largely used, mostly in legacy
systems, it seems clear it will not play a significant role in new developments. Among
its advantages, DCE has tested support for interoperability in heterogeneous
environments and mature services that increase systems reliability. However, there are
also great disadvantages for recommending its use:
56
•
•
•
•
There is not practical support for object-oriented languages.
Weak market position in relation to CORBA and DCOM.
DCE provides rather low-level services.
There is no component market for DCE.
When looking for a middleware technology for ICa, some years ago, DCE was the
initial election. At the end, it was replaced by CORBA, then emerging, for the above
reason concerning object-orientation that was deemed critical for the ICa
infrastructure.
3.5 OMG’s Distributed CORBA Architecture
The Object Management Group is the largest software consortium in the world. It has
more than 500 members all of them related in one way or another to the computer
industry. The OMG work began back in 1989 founded by just eleven companies. It is
committed to the development of vendor-independent technical specifications for the
software industry. Basically, its primary mission comprehends the creation of a
marketplace for component software and systems by the establishment of industry
guidelines and object management specifications. The Common Object Request
Broker Architecture (CORBA) is it most famous specification, together with UML,
that deals with distributed object-oriented middleware. CORBA instances the
interfaces for the entities defined in the Object Management Architecture (OMA), the
OMG’s reference architecture for distributed object systems.
CORBA is already a mature technology, created before DCOM and after DCE. It has
always been object-oriented and vendor, language and platform independent.
CORBA’s objective is that of the OMA, to enhance interoperability and integration
among distributed systems. As other types of middleware, CORBA uses an Interface
Definition Language for the specification of objects interfaces. In the case of OMG or
CORBA IDL, this language has been standardized as an ISO standard (ISO / IEC DIS
14750).
The CORBA specification will be dealt with in depth in the following chapters.
Nevertheless, it is worth to enumerate some of its advantages in this section.
•
•
The CORBA specification is open. The specification follows an open process
for its evolution or for the creation of new specifications.
CORBA is object-oriented from inception. However, it can also be used to
encapsulate non-object-oriented systems as it defines mappings for a huge
57
•
•
•
•
•
•
amount of languages. This means that CORBA is able to integrate well with
legacy applications and systems.
CORBA has support for an overwhelming number of different platforms and
operating systems, being the heterogeneous environments its natural habitat.
CORBA is supported by hundreds of vendors.
CORBA and the OMA provide support for the concept of services, facilities
and for domain specification which improve system development and
reliability by the use of standard middleware components.
The ORB concept provides transparency to the distributed system in several
ways. Location transparency, access transparency, concurrency transparency,
migration transparency, etc.
CORBA emphasizes dynamic interface availability when needed. CORBA
provides a Dynamic Invocation Interface (client side) and a Dynamic Skeleton
Interface (server side) that confer greater component mobility than other types
of middleware.
CORBA is easier to learn and has a cleaner object model than DCE, Java /
RMI and DCOM. DCE, apart from not being object-oriented, suffers from
being too close to the operating system and developers are worried about
details that have nothing to do with the application functions. On the other
hand, DCOM is based on complex C++ virtual method calls, which yields
complex interfaces usually generated by specialized tools. This makes
complex to understand large systems.
Regarding the last item in the previous list, CORBA has always been criticized as a
difficult to learn technology. As happens with most things, CORBA can be a difficult
or easy to learn technology if compared to other options. Easy or difficult is not an
absolute magnitude neither is it something that can be measured objectively.
Regarding DCE, Java / RMI and DCOM, CORBA is easier to learn than the other
three. As an example take the following IDL excerpt of an interface for the three
object-oriented types of middleware. The IDL code specifies a sensor interface with
an operation to read its value.
DCOM IDL
CORBA IDL
Java/RMI IDL
[
uuid(7371a240-2e51-11d0b4c1-444547850000),
version(1.0)
]
library
Sensors
{
importlib("stdole32.tlb");
[
uuid(BC4C0AB0-5A45-11d2-
module
Sensors
{
interface
ISensor
{
float get_ value (
in string varName
);
};
};
package
import
import
58
Sensors;
java.rmi.*;
java.util.*;
public interface Sensor
extends
java.rmi.Remote
{
float get_value(
String varName
) throws RemoteException;
99C5-00A76814C655),
dual
]
interface
ISensor
:
IDispatch
{
HRESULT get_value(
[in] BSTR varName, [out,
retval]
float
*
rtn);
}
}
[
uuid(BC4C0AB3-5A45-11d299C5-00A02374C655),
]
coclass
Sensor
{
interface
ISensor;
};
};
File: SensorLib.idl
File: Sensor.idl
File. Sensor.java
3.6 Microsoft’s Distributed Component Architecture
The Distributed COM (DCOM) extends the functionality of the Component Object
Model (COM) to support inter-host communications. In COM, the interaction between
clients and components is defined without the need of mediating middleware or any
other system component. But, as in the operating system processes run in virtual
address spaces, it is not possible to directly communicate with a component of a
different address space. For this reason, COM uses an inter-process communication
mechanism (LPC or Local Produre Call) to allow clients access the components out of
its address spaces (see Figure 21).
Client
COM
Run-time
Security
Provider
DCE
RPC
COM
Run-time
Security
Provider
LPC
Component
DCE
RPC
LPC
Figure 21: COM components running in intra-host processes
As can be seen in the figure, COM relies on the DCE RPC mechanism to forward
requests between clients and servers. In the case where client and component reside in
different hosts, the previous mechanism is not valid. The obvious solution has been to
59
replace the local inter-process communications with a network protocol that allows
COM components to be distributed over a network. As in other types of middleware
this is done without any of the involved endpoints noticing that they are placed in
different locations. The DCE RPC and the security provider layers of COM are used
to generate network packets that conform to the DCOM wire-protocol specification
(see Figure 22).
Client
COM
Run-time
Security
Provider
COM
Run-time
Security
Provider
DCE
RPC
Protocol
Stack
Component
DCE
RPC
Protocol
Stack
DCOM networkprotocol
Figure 22: DCOM inter-host architecture
The figures show a simplified architecture, as the RPC must be object-oriented an
additional layer among the COM run-time and the remote procedure call is used to
match objects and RPCs and to resolve object identities. From this point of view and
from that of the DCOM architecture, it is more an application of the DCE technology
than a truly new approach to distributed object computing. As COM is object-oriented
but DCE is not (although it has some extensions for it), the mix results in complex
proprietary interfaces to distributed systems. This is one of the reasons why tools are
necessarily employed to generate DCOM interfaces. In platforms (UNIX and OS/390)
where DCOM has been ported but where there is not good support for development
tools, the problem of developing interfaces becomes a hard to solve one.
As with other types of middleware, DCOM has some advantages and disadvantages.
•
•
DCOM has a large user base and component market.
DCOM is well suited for desktop applications but not for enterprise
applications.
60
•
•
•
3.6.1
Middleware is free in Microsoft Windows platforms. It comes bundled with
the operating system but outside the Microsoft world there is minimal
availability for DCOM.
The programming style of DCOM is complex. The interface specification is
complex as it is a revamp of earlier technologies such as OLE.
Deciding on DCOM is deciding on Microsoft. DCOM, with a few exceptions,
only runs over the MS Windows operating systems and supported hardware
platforms. There is no interoperability in heterogeneous environments.
.NET Framework
In the rapidly changing world of Microsoft nothing is built to last. After DCOM was
built upon OLE and COM, Microsoft kept on producing new developments upon the
previous infrastructure. The first of these developments was MTS, the Microsoft
Transaction Server which embeds the functionality of a transaction monitor and that of
an ORB for components. The name of MTS is badly chosen because it is more than a
transaction processing monitor. After MTS, they announced COM+. A good definition
of COM+ is the following.
COM+ = COM + DCOM + MTS + MMQS + Services. [Pascual 00]
MMQS stands for the Microsoft Message Queue Server. The additional services
provided by COM+ include load balancing between component servers and a new
event model. COM+ is an extension of the functionality of DCOM and MTS that is
identified with Microsoft’s DNA; the Distributed interNet application Architecture
that is a methodology for building distributed applications. DNA was the forerunner of
.NET; the last initiative from Microsoft that introduces a completely new object model
based on the eXtensible Markup Language. All this has occurred in the last 6 years.
Certainly, IT managers of process plants would feel their nerves crisping if they
follow and implement all Microsoft’s initiatives. In the industry, systems are expected
to have a long lifespan and to evolve with an integrative approach.
The architecture for .NET is a multi-tier (3 tier) client / server model where
applications are built by composition of web services that work together to provide the
application’s functionality. Again, everything is based upon the COM and DCOM
infrastructure but with a great difference. In the .NET model web services are black
boxes that can be accessed only via HTTP and XML. Web services can not be
accessed via IIOP, DCOM or RMI. This point clearly makes the .NET architecture not
usable in the industrial environment where there are strict requirements regarding
61
performance (at least the web services concept). In .NET messages are represented in
XML and are sent to web services in SOAP format (Simple Object Access Protocol)
via HTTP.
3.6.2
OLE for Process Control
OLE for Process Control or OPC for short, its backed by the OPC Foundation which is
supported by Microsoft. The foundation has more than 300 members of the
automation industry. The OPC Foundation objective is creating and maintaining open
specifications (sic) that standardize the communication of acquired process data, alarm
and event records, historical data and batch data to multi-vendor enterprise systems
and between production devices (PLCs, RTUs, DCSs, etc.).
OPC was originally based on the OLE, COM and DCOM specifications and defines a
standard set of objects, interfaces and methods for use in process control applications.
The first OPC specification tried to address the problem of data access and now is
called the Data Access Specification. As of today, OPC counts with many more
specifications regarding security, alarms and events, historical data, etc.
The problem that OPC tried to initially solve is what is called the I/O Driver problem
(see Figure 23). If each software driver presents a different interface to applications,
then the application needs a specific piece of code to access data from each driver.
Display
Application
Software
Driver
Trend
Application
Software
Driver
Software
Driver
Report
Application
Software
Driver
Figure 23: The I / O driver data access problem
62
The OPC foundation solved the problem by creating a software bus. If all the involved
applications (OPC clients) and device drivers (OPC servers) speak OPC a software
bus is created and data can be accessed uniformly across the process control
application. This means all blue, red, green and gray rectangles of the figure become
of the same color, needing then only a single piece of code (OPC) to communicate
with all devices. As the main drawback of the technology, OPC is built upon COM
and suffers from the same advantages and disadvantages of it with one difference.
OPC can run in more heterogeneous hardware (at the device level) than its COM
counterpart. This is possible, due to the interest of device vendors in providing OPC
servers with their products.
As with other technologies from Microsoft, OPC and COM have not been initially
thought as part of a real-time system. This means that OPC has no support for realtime features and lacks the concept of deadline or priority in the worst of cases. It is
true that it supports time-stamping but there are no guarantees as to when the timestamped value will be obtained. From the OPC client-side point view and for some
process control applications, Microsoft Windows is not a real-time operating system
so COM and OPC who are sitting over the OS cannot also offer real-time capabilities.
The final result is simply that OPC cannot cope with all the demands of present
process control systems. Without disregarding these considerations, the OPC
technology proves useful in those applications where the overhead introduced by OPC
is less than the time constants of the controlled systems, typically these are systems
with large inertia like steam boilers and akin processes.
Whereas OPC is a useful technology for some kind of processes it does not cover the
full range of requirements present in control applications. In the next chapters, the way
in which CORBA-based technology can address all these requirements will be
demonstrated.
3.6.3
Microsoft’s View of DCOM and CORBA
Microsoft recognizes in its own documents that CORBA is a competing specification
for distributed object computing. Microsoft, knowing one of CORBA’s weakest
points, criticizes its lack of a binary format specification as one of the reasons why
there is not a market for off-the-shelf reusable components. In words of Microsoft, this
means that component providers have to supply source code for their components or
compile and test not only for each target platform but also for each target ORB
implementation.
63
Although, the claims of Microsoft are true they do not hide the fact that Microsoft
suffers this very same problem. It is true they have a binary format, but it is their own
binary format that only runs on one single platform. No consensus is reached with any
other manufacturers and, as our premises include that we are addressing
heterogeneous systems in nature, it results that DCOM simply can not interoperate in
this kind of environment (further Microsoft Windows only runs on Intel hardware 2 ).
On the other hand, much has been gained from former CORBA implementations in
what refers to compliance to the specification and libraries of components can be
provided in object code form for each platform.
The second claim from Microsoft is related to broker interoperability. They claim that
CORBA components will only be able to run over a predetermined ORB. Microsoft
defends the position that ORB specific and proprietary extensions sacrifice cross-ORB
interoperability and thus the ability to develop components for several ORBs with a
moderate effort. Although it is also true that most implementations provide extensions
in the form of helpful gadgets (most in the form of language macros for redundant
tasks), these extensions do not impede interoperability, one of CORBA major
achievements. In the author’s experience developing CORBA components, it has been
possible to run exactly the same software implementations over the following ORBs:
Borland’s Visibroker, Washington University’s TAO, IONA ORBIX, Vertel’s e*ORB
(now bought by PrismTech), OIS Orbacus and SCILabs’ ICa. The only task needed
for this level of interoperability consists in recompiling the component IDL files for
each ORB implementation to generate the object’s stubs and skeleton whose inner
structure is not standardized by the CORBA specification. This is certainly much more
regarding component interoperability than what Microsoft ever offered.
3.7 Sun’s Java Distributed Component Architecture
Java / RMI is the basic infrastructure for Sun’s distributed component architecture.
Java is an interpreted, object-oriented language designed in its origin to be used in
distributed applications and to be portable across platforms. For portability, Java relies
on the concept of virtual machine. All Java software runs, theoretically, over the same
software platform (the Java Virtual Machine) that sits on top of the operating system.
Distribution in Java is achieved by means of Remote Method Invocation or RMI that
An exception is Microsoft Windows Embedded (CE and XP Embedded) which
supports other CPU architectures (e.g. ARM, MIPS, SH4, etc.) from different
manufacturers.
2
64
allows objects residing in one virtual machine to access objects residing in another
virtual machine.
RMI, the invocation mechanism in Java, is a client / server model for distributed
applications. Since its inception one of the central or unique features of RMI has been
its ability to download the bytecodes of an object’s class if the class is not defined in
the receiver’s virtual machine. This means that new types can be introduced in a
remote virtual machine being it possible to change the behavior of an application
dynamically. The RMI architecture consists of three different layers (see Figure 24).
•
•
•
The Stubs and Skeletons layer. This layer implements the client-side stubs and
server-side skeletons needed for communications. Basically, the functionality
of this layer comprehends the marshalling an un-marshalling of method
arguments and the notification to the Remote Reference Layer that a new
request is ready. The behavior upon reply return from the server object is
reverse to that explained.
The Remote Reference Layer. This layer implements the remote reference
protocol independently of stubs and skeletons (e.g. invocation to a single
object or to replicated objects) and transmits the request to the transport layer
in a stream-oriented connection abstraction.
The Transport Layer. The transport layer is in charge of the setting up and
management of connections and to carry out object tracking.
As in CORBA, clients making use of remote objects need a proxy or stub that acts on
behalf of the remote objects. The object is invoked as if it were a local object.
Application
RMI System
Client
Server
Stubs
Skeleton
Remote Reference layer
Transport
Figure 24: RMI architecture
It is worth noticing that the RMI only interoperates among objects written in Java
language so it is language dependent. In fact, in the Sun’s component model
represented by the JavaBeans (an extension to Java objects that is a first component
model designed for GUI and desktop objects) and by its extension EJB (the newer
65
component model Enterprise JavaBeans) reliance is put on IIOP if communication is
needed with objects written in other languages. For this, the OMG defined a Java-toIDL binding that enables direct interaction with an ORB. JavaBeans are utilities that
are client-side bound. In the component model of Sun there was need also to address
the server-side with components that support features like load balancing or resource
pooling. This was a must as distributed applications based in the component model
need to scale well and Sun produced the EJB specification to address these issues. EJB
was initially a specification not an implementation and Sun relied on third parties to
provide implementations.
Initially, Sun named the Java Class Libraries and the Java Virtual Machine (JVM) as
the Java Development Kit (JDK). In 1999, the JDK was extended and it was renamed
after Java 2 Platform. There are different versions of the Java 2 Platform depending on
whether the applications are for clients, servers or mobile devices. The most popular
Java platform has become J2EE, the Java 2 Platform Enterprise Edition (see Figure
25).
Application Programming Model
EJB
Java Server
Page
Transactions
Messaging
Mail
Servlets
Applets
Container
Connectors
JavaBeans
Tools
CORBA
RMI
Database
Naming /
Directory
Figure 25: The J2EE architecture
Among the disadvantages of Java concerning the development of distributed control
systems, there are the following among others.
66
•
•
It is very slow compared to CORBA. This is true if compared to other
programming languages, C++ for example. The statement also holds when
compared to other types of middleware (RMI has a poor performance
compared to CORBA).
Java cannot directly access operating systems and hardware functions. It runs
on its own private virtual machine so non-Java access to lower layers of the
virtual machine needs to be coded in the Java Native Interface (JNI) 3 .
3.8 Comparing Apples to Apples
Throughout this chapter, we have discussed about the taxonomy or classification of
the different types of middleware available in the market. Also, there has been made a
distinction between objects or distributed objects and components and it has been
mentioned that almost all types of middleware support a certain component model.
Even there are cases as that of Sun where more that one component model is used.
Table 2 summarizes these concepts.
Software
Properties
Distributed
Middleware
Object Model
Component Model
Advanced
Component Model
Data Access
Messaging
Naming and
Directory
OMG
Sun
Microsoft
CORBA
RMI, CORBA
DCOM
CORBA
CCM
CCM
Java
JavaBeans
EJB
COM
COM, COM+
MTS (COM+)
Not available
JDBC
Event, Messaging
specification
Naming Service
Trading Service
JMS
ADO, OLE DB,
ODBC
Event Server,
MSQM
Active Directory
JNDI
Table 2: Classification of middleware features
In the table above, the different overall abilities of each type of software are listed.
One could be tempted to compare the EJB with the CCM component models but it
An exception is that real-time versions of Java allow to interact with physical
memory.
3
67
would render invalid results unless trying to reach very general conclusions as the
target focus of each component model is pointed towards different uses. Simply put,
an apple (component model) to apple (component model) can not be simply done. In
the scope of this work, we are worried about industrial distributed control systems
based on a common middleware technology no matter the layer of control where the
middleware should work. Enterprise-oriented component models (EJB) and COM+
are less interesting for us than the OMG’s one as the properties of the enterprise we
are considering are different from those of the former models. In some of the control
layers, components could be used but in others only distributed objects can be
employed. The main reason for this is the lack of resources in the lower layers of a
control system. These layers are those of the basic control loops and field devices with
limited capabilities.
CORBA has a great advantage over Java and DCOM as it has specialized
specifications for vertical domains as that of the industry. For instance, the following
are some of the available specifications in the industrial domain.
•
•
•
•
Real-Time data distribution [OMG 03e].
Data acquisition from industrial systems [OMG 02c].
Historical data acquisition from industrial systems [OMG 03f].
Utility management systems data access facility [OMG 02d].
Even more, the current ongoing research on real-time component models focuses
mainly into the CORBA world as Microsoft is not interested in real-time and the
development on distributed real-time Java has been stopped for a long time now.
68
Chapter 4
CORBA Systems
69
4.1 CORBA Technology
This chapter introduces the basics of the technology employed for CORBA Control
Systems. The OMG has produced and continues to evolve a huge amount of
specifications related to CORBA. Many of them are interesting to the distributed
control field while others are more focused on business or enterprise applications. In
the next sections, the most relevant aspects and topics of CORBA technology for
distributed control applications are presented. There is information in this chapter
about the following topics.
•
•
•
•
•
•
•
•
The Object Management Group (OMG). As the organization leading the
CORBA and other important specifications (e.g. UML) and standards (e.g. the
IDL standard).
The Object Management Architecture (OMA). As the reference
architecture for CORBA Control Systems.
CORBA Object Services (CORBAservices). As the basic building blocks of
CORBA Control System distributed applications.
The CORBA Component Model (CCM): As the next step in future CORBA
applications.
Real-Time CORBA. As the underlying framework for distributed
applications in which control of resources can be exercised for end-to-end
predictability.
Minimum CORBA. To be employed in resource-limited devices of control
systems.
Data Distribution Service. For the dissemination of information in a
distributed system with real-time requirements.
Data Acquisition from Industrial Systems (DAIS) and Historical DAIS.
Describes and interface for data, event and alarm access from industrial
systems. The HDAIS specification defines an API for its use with time series
data in industrial systems.
4.2 The OMG
The Object Management Group (OMG) is a not-for-profit organization established in
1989 with the goal of developing vendor independent specifications to foster object
technology by means of the creation of a software marketplace for object technology.
It aims to reduce the complexity, lower the costs, and hasten the introduction of new
software applications and systems. By using OMG's object technology any
71
organization can leverage previous efforts in building distributed object systems. The
OMG is a standardization organization with an open, vendor-neutral, international,
widely recognized and rapid standardization process based on demonstrated
technology.
Nowadays, software vendors, developers and users working in various different fields
belong to the OMG membership as well as universities and governmental institutions.
Further, it maintains a strong liaison with other organizations like ISO, ITU-T, W3C,
TINA-C, and Meta Data Coalition.
OMG's object technology is the object technology of reference: CORBA, IDL, UML,
MOF, XMI, MDA, etc. Their standards allow interoperability and portability of
distributed object oriented applications of different vendors. They do not produce
software or implementation guidelines; only specifications which are put together
using ideas of OMG members who respond to Requests For Information (RFI) and
Requests For Proposals (RFP). The strength of this approach comes from the fact that
most of the major software companies interested in distributed object oriented
development are among OMG members.
4.3 Brokering Architectures
Brokering architectures are those based on an Object Request Broker (ORB). In words
of the OMG, an ORB is the middleware that establishes client-server relationships
between objects. By means of the ORB, clients can transparently invoke methods on
objects, which can be on the same machine or across a network. ORBs receive client
calls and it is under their responsibilities to find the object that implements the
requests, pass it the request parameters to invoke the method and to return the reply if
any.
For the clients there are several levels of transparency. The client is not aware where
the object providing the service is located or the programming language it is
implemented in. Neither it is aware of the operating system or network protocol used
to access the object. The client is only aware of the object’s interface, which is all
needed to access the object.
The ORB is responsible for resolving all the details regarding how the object is
reached, providing in this way interoperability in heterogeneous distributed
environments. For interoperability to be possible between different hardware devices,
programming languages, operating systems and networks an interoperability protocol
independent of all the former conditions is needed. In an ORB, the interoperability
72
protocol is simply defined by the application interfaces which are expressed in a
language-independent interface specification language, the IDL. IDL is good because
it concentrates on the important part, what the offered services are. In the case of
OMG’s IDL, the ISO standard IDL, it is as CORBA object-oriented. But this does not
mean integration is only possible within an object-oriented framework. One of the first
objectives of an ORB is integration. Within an ORB, integration is possible even with
old legacy systems which are not object-oriented (let’s say a COBOL system) because
the only thing IDL specifies is the interface. It does not specify how the system is
implemented. Legacy systems can be integrated into and ORB-based or in a brokering
architecture by the use of interface wrappers. Further, the OMG defines CORBA
mappings to non-object-oriented languages.
The most worldwide spread brokering architecture is OMG’s OMA, the Object
Management Architecture from the OMG which is treated in the next section.
4.4 Object Management Architecture
The Object Management Architecture (OMA) [OMG 96a] belongs to the main
contributions of the OMG to the OO world. This is a specification for the construction
of open distributed object systems based on brokering and on a collection of
predefined services. It is a high-level vision of a complete distributed environment and
consists of four components that can be roughly divided into two parts: system
oriented components (Object Request Brokers and Object Services), and application
oriented components (Application Objects and Common Facilities).
The items below describe the main building-blocks of the OMA:
•
•
•
Object Request Broker (ORB): the central building block of the OMA that
is the run-time integration vehicle for forwarding requests and responses
between CORBA objects.
Common Object Services (now called CORBAservices): provide
fundamental services that are near to the system level (e.g. Naming Service
and Notification Service).
Horizontal CORBAfacilities: services that do not fit into a particular vertical
market but are at a too high level to be called CORBAservices (e.g. the
Printing Facility, the Secure Time Facility, the Internationalization Facility,
and the Mobile Agent Facility).
73
•
•
Vertical CORBAfacilities (or Domain CORBAfacilities): provide
standardized services for a particular field of application (e.g. Healthcare and
Transportation, Utilities or Telecommunications)
Application Objects: provide a particular service for a specific customer.
The Object Request Broker (Figure 26) is the one of these parts which constitutes the
foundation of the OMA and manages all communication between its components. It
allows objects to interact in a heterogeneous, distributed environment, independently
of the platforms on which these objects reside and of the techniques and languages
used to implement them. In performing its task it relies on Object Services which are
responsible for general object management such as object creation, access control, etc.
and to provide an environment in which single objects perform their tasks.
Common Facilities and Application Objects are the components closest to the end
user, and in their functions they invoke services of the system components. These
types of facilities can be configured to the purpose of a specific application.
Application
Objects
Vertical
CORBA Facilities
Horizontal
CORBA Facilities
Object Request Broker
CORBA Services
Figure 26: The CORBA OMA
When the OMG designed the OMA, it had to establish a balance, from the software
architecture point of view, among its quality attributes [Bass 98]. The basic attributes
it tried to address are:
•
•
Buildability. The OMG tried to specify an architecture which could be
feasibly implemented with available technology. Buildability is managed by
the OMG standardization process that requires commercial implementations
of the specification be built within one year of their approval.
Balanced specificity. The architecture should be detailed enough to provide a
standard that could be implemented by developers but general enough to
74
•
•
•
•
avoid vendor lock-in and to allow market competence by optimizations and
specific features. This attribute is represented by the OMA ability to represent
object interfaces in IDL.
Implementation transparency. The architecture should provide transparency
from implementation details so application clients can be independent of
object implementation details.
Interoperability. This is addressed by the interoperation of objects developed
on OMA implementations from different vendors and by the implementation
of bridges to other technologies.
Evolvability. The architecture should be able to accommodate new distributed
system technologies and needs from the market (this is achieved by the object
services and facilities of the OMA). OMG’s Task Forces and special-interest
groups are in charge of the evolvability process of the OMA.
Extensibility. The architecture provides a nucleus of features which are useful
and needed by most distributed application developers but is open enough to
accommodate specific niche features (room for extensibility is provided by
vertical domain facilities).
The OMA provided a balance by making a trade-off of all the above attributes which
are in tension by each other; a general but detailed architecture, general but niche
optimal, stable versus flexible and so on. The trade-off performed on the quality
attributes of the OMG brokering middleware is reflected on how the OMA is
partitioned into architectural elements. CORBAservices are useful to most developers
while specific domain vertical facilities are good for specific niches. All these services
are independent of those of the ORB which are needed by services and facilities
(which also are CORBA objects) and from application objects which are application
specific.
4.5 Services and Facilities
Of special interest are the CORBAservices to all distributed applications. Services
provide pre-built functionality for the construction of applications from CORBA
building blocks. They are classified as CORBA services as they are useful in a wide
range of different domains (horizontal services) and should be used whenever possible
to enhance application reliability. Next a description of some of the most interesting
CORBA object services is presented.
•
Collection Service: The Collection Service [OMG 02e] provides a uniform
way to manage the most general types of collections. Collections are
groupings of objects and the operations needed to manipulate them as a group.
75
•
•
•
•
•
•
The Collection Service supports three types of interfaces in order to manage
collections; collection and collection factory interfaces, iterator interfaces and,
function interfaces.
Concurrency Service: The purpose of the Concurrency Service [OMG 00] is
to mediate concurrent access to an object such that the consistency of the
object is not compromised when accessed by concurrently executing
computations. The Concurrency Service uses a Lock Model in order to
guarantee serialized access to objects.
Enhanced View of Time: This specification [OMG 02a] leaves the earlier
Time Service Specification [OMG 02f] unchanged and presents a service in
which clocks with properties different of those of the Time Service can be
used. The main advantage of the specification is its ability to represent clocks
with differing characteristics (resolution, precision, stability or offset, skew
and drift if more than one clock is used in the system).
Event Service: There are situations in which it is necessary to decouple
CORBA clients and server objects. The Event Service [OMG 04] provides the
roles of event supplier (data producer) and event consumer (data processor) in
order to provide decoupling between objects. Data is communicated between
suppliers and consumers by issuing CORBA requests. The service supports
two event communication models. The push model and the pull model. In the
push model a supplier of events is able to initiate the communication of data
to the consumers. In the pull model the consumer initiates the communication
of the event. An event channel is a CORBA object that allows asynchronous
communication of events between multiple suppliers and consumers by using
standard CORBA requests.
Externalization Service: The Externalization Service [OMG 00a] provides
conventions and protocols for externalizing and internalizing objects. To
externalize an object is to record the object’s state in a stream of data. Clients
can request that externalized data be stored in a file using a standardized
format that is defined as part of the Externalization Service specification.
Internalizing is creating a new object from the data stream of an externalized
object.
Licensing Service: The Licensing Service [OMG 00b] provides a mechanism
for producers to control the use of their intellectual property in a manner
determined by their business and customer needs. It is important to notice that
no products with temporary or expiring licenses can be selected for mission
critical applications.
Life Cycle Service: The Life Cycle Service [OMG 02g] provides a set of
interfaces for creating, deleting, copying and moving objects. The service
76
•
•
•
•
•
•
allows clients to perform life cycle operations on objects residing in different
locations.
Lightweight Services: These are specifications of services for resource
constrained systems. There are Naming, Event and Time services lightweight
specifications [OMG 04a] and a Lightweight Log Service [OMG 03g]. The
log service is designed as a central facility inside an embedded or real-time
system capable to accept and manage logging records. Records are stored in a
memory area and are owned and managed by the Lightweight Logging
Service.
Naming Service: The Naming Service [OMG 02h] allows for humanreadable naming of objects. Names can be associated to object IORs (binding)
and object references can be obtained by the use of the object name
(resolved). The Naming Service can be thought of as a “yellow pages” service
for CORBA objects. It is based upon the name-to-object association which is
called name binding. Name bindings are contained inside Naming Contexts
which are CORBA objects that manage a set of name bindings. Each name
binding is unique inside a naming context.
Notification Service: The Notification Service [OMG 04b] extends the Event
Service providing new capabilities. Among these new capabilities are the
functionality to transmit events in the form of well-known structures, the
ability for clients to receive specific events, the ability for suppliers to inquire
about the existence of consumers so they can provide events on demand (this
puts some leverage to the bottleneck problem that can occur when too many
suppliers and consumers use an Event Service).
Persistent State Service: The Persistent State Service [OMG 02i] provides
interfaces for the retention of the persistent state of objects. This service goes
further than the object reference persistent storage that CORBA provides
(object_to_string) and should be capable of restoring the dynamic state of an
object from its persistent state. The Persistent Object Service uses the
Externalization Service as a POS (Persistent Object State) protocol.
Property Service: The Property Service [OMG 00c] provides support to
dynamically associate objects to properties which are typed named values
(tuples <name, value>). The object properties can be used for multiple
purposes (e.g. object classification regarding some property). Properties are
not part of the object type; instead it is information of an object maintained by
the service. Properties are the dynamic equivalent of CORBA attributes.
Query Service: The Query Service [OMG 00d] provides query operations on
collections of objects. Queries are predicate-based and may return collections
of objects. The collections returned by the Query Service may be selected
from a source of collections based on whether their member objects satisfy a
77
•
•
•
•
•
•
given predicate or may be produced by query evaluators based on the
evaluation of a given predicate.
Relationship Service: Objects are frequently used to model entities of the
real world. Thus, objects do no exist alone but related to others. The
Relationship Service [OMG 00e] allows entities (CORBA objects in a
distributed system) and relationships to be explicitly represented.
Security Service: The Security Service [OMG 02j] enables a distributed
CORBA system to have control over identification, authentication,
authorization and access control. It also provides interfaces for security
auditing, administration, security of communication between objects and nonrepudiation functionality.
Telecom Log Service: The Telecom Log Service [OMG 03h] extends the
present Log Control Function specification (ITU-T X.735/ISO 10164-6) of the
International Telecommunication Union which allows logging any type of
event and querying of the log records by the use of a constraint language.
Time Service: The Time Service [OMG 02f] enables its clients to obtain
current time along with an estimate of the error associated with it.
Unfortunately, the time representation (UTC from the X/Open DCE Time
Service) of the Time Service Specification is not well suited for some fields of
application. Other specifications as the Smarts Transducers Specification
make better choices as their representation of time is synchronized with that of
GPS and have a granularity of up to 60 ns.
Trading Object Service: The Trading Service [OMG 00f] enables to offer
and to discover instances of services of particular types. A trader is an object
which supports the Trading service and can be considered as a whiteboard
where other objects can either advertise their capabilities or search for objects
which advertise certain features.
Transaction Service: The Transaction Service [OMG 03i] defines interfaces
that allow multiple, distributed objects to collaborate to provide atomicity.
The service interfaces allow the objects either to commit all changes together
or to rollback all the changes together.
4.6 Real-Time CORBA
There are a great number of distributed systems which are real-time systems. These
are systems which have requirements regarding the end-to-end predictability and the
Quality-Of-Service (QoS) of their activities. Examples of this type of systems can be
found in the process control, command and control, avionics or telecommunication
fields among others. If advantage is to be taken from the benefits of using an ORB for
78
the development of real-time distributed applications, the requirements for that type of
systems must be taken into account at the middleware level.
RT CORBA [Schmidt 00] specifies additional mechanisms to those of CORBA that
can be employed to increase the control of resources and end-to-end predictability of
distributed object applications. Figure 27 provides an overview of the extensions
compared to traditional CORBA. The following sections provide a description of the
key entities and features of a real-time CORBA broker as addressed in the Real-Time
CORBA specifications (see [OMG 05] and [OMG 03]). There are currently two
specifications for real-time; one addresses fixed priority scheduling [OMG 05] and the
other addresses dynamic scheduling [OMG 03]. Although an overview is given for
both of them, in practice, most real-time ORBs conform to the fixed priority
scheduling specification.
Scheduling Service
Client
Server
RTCORBA::Current
POA
RTPOA
CORBA::Current
RTCORBA::Priority
RTCORBA::Threadpool
ORB + RTORB
IIOP
RTIOP
RTCORBA::PriorityMapping
Figure 27: Real-Time CORBA extensions provide strong
control over application resources on the client and server
sides
4.6.1
Fixed-priority Real-Time CORBA
The first real-time CORBA specification did not address the problem of dynamic
scheduling. It only supports fixed-priority scheduling. The objective of the
specification is the control of resources to enhance end-to-end predictability in a
distributed real-time system. Regarding end-to-end predictability the specification is
defined to mean:
• Respecting thread priorities between clients and servers for the resolution of
resource contention during the processing of CORBA invocations.
• Bounding of thread priority inversions during end-to-end processing.
79
•
Bounding the latencies of operations invocations.
The specification relies in an extension of key CORBA entities (ORB, POA, Current,
threading model, etc.) and on the services provided by them. The Real-Time CORBA
specification allows managing three types of resources in a distributed system.
•
•
•
4.6.2
Processor resources.
Communications resources.
Memory resources.
RT-ORB
The real-time ORB exploits an extension of the CORBA::ORB interface that provides
operations to create and destroy constituents of a real-time ORB. A real-time ORB
incorporates functionality that allows the application running on top of it to specify the
information related to QoS or resource requirements needed for the correct operation
of the system. For example, it is possible to configure the range of thread priorities the
ORB is allowed to use during execution.
4.6.3
Real-time POA
In much the same way a real-time ORB must be able to process the real-time
requirements of the objects on top of it, the Object Adapters running on its server side
must also understand how to process those requirements. The Portable Object Adapter
for Real-time CORBA is defined in the module RTPortableServer. The Real-Time
POA is a subtype of the CORBA PortableServer::POA. The POA for real-time has
two different characteristics from that of CORBA:
•
•
It should understand the real-time policies specified in the real-time extension.
It provides an additional set of operations to support object level priority
settings. The Real-Time POA groups a set of operations designed to override
the server declared priority on a per-object reference basis. Examples of these
operations are create_reference_with_priority or activate_object_with_priority
that allow setting an execution priority for CORBA objects or interoperable
references.
A common behavior of real-time POAs is to encapsulate the values regarding to
server-side real-time policies into interoperable object references (IORs) so it is
possible for the clients accessing those objects to honor the real-time behavior
expected by them.
80
4.6.4
CORBA Priorities
Different RTOSs show different native thread priority schemes. As a result of RTOS
heterogeneity, different native priorities exist which represent a problem when trying
to establish a common behavior in a real-time heterogeneous distributed system. RealTime CORBA solves this hindrance by defining a CORBA Priority type which is
valid for the entire system. As in the RTOS native priority schemes, CORBA Priority
is a set of integer values that maps the native priority scheme of a specific RTOS to a
uniform scheme which is accepted system-wide. The Priority for real-time is defined
in the RTCORBA module.
Real-Time CORBA also defines a PriorityMapping object which is defined as an IDL
native type. Native types are a special type definition in CORBA that result in specific
implementations in each programming language. A PriorityMapping is a programming
language object (an instance of the RTCORBA::PriotityMapping class in C++) instead
of a CORBA object. Consequently, mappings to programming languages (C, C++,
Ada and Java) are also provided in the submission for Real-Time CORBA. Figure 28
shows a mapping of priorities between Real-time CORBA and two particular
operating systems. Notice that not all the priorities have to be mapped from CORBA
to the OS. The priority mapping interface allows to perform the mapping of CORBA
priorities to the native values of the operating system and from the operating system to
CORBA.
RTCORBA::Priority
31
0
0
31
25
4
255
OS #1 native priority model
0
OS #2 native priority model
Figure 28: CORBA Priorities and mapping to
OS
CORBA priorities are used to allow the clients and CORBA objects specify the
priorities at which requests must be executed. This allows enhancing end-to-end
predictability in a fixed-priority scheduling mechanism.
81
4.6.5
Real-time Current Interface
Real-Time CORBA derives the real-time Current interface from the CORBA::Current
interface. The specific objective of Current for real-time is to obtain the CORBA
Priority of the current thread. The RTCORBA::Current object contains a priority
attribute which can be set and consulted. As a result of setting the priority attribute,
the ORB sets the native priority of the thread to the value returned by
PriorityMapping::to_native. With this feature, the process under execution can
dynamically act upon the thread priority it has been assigned by the ORB.
4.6.6
Real-time CORBA Priority Models and Transforms
In the context of a real-time distributed system there must be resources that allow the
infrastructure to enforce thread eligibility of execution on remote objects. For this
purpose, Real-Time CORBA supports two types of priority models to coordinate
priorities across systems (see Table 3). The Priority Model along with the use of
CORBA Priorities is designed to minimize end-to-end priority inversion as well as to
bound latency and jitter for applications with real-time requirements.
Priority Model
Client Propagated Priority Model
Server Declared Priority Model
Description
Priority is carried with the CORBA invocation and is
used to ensure that all threads subsequently executing
on the invocation run at the appropriate priority.
In this model, objects publish their CORBA priorities
in their object references. This lets clients know the
priority of invocation execution in the servant’s code.
The Real-Time CORBA Priority of an invocation
needs not be passed from client to server in an
invocation.
Table 3: Real-Time CORBA Priority Models
Additionally, it is possible to override a Server Declared Priority on a per-object
reference basis. The Real-Time CORBA POA interface provides four operations for
doing this. The server priority can be changed at the time of object reference creation
or at the time of object activation.
Priority Model Policies must be applied to POA objects at the time of creation. To let
clients know the policy models supported by CORBA objects, the policies are
propagated from servers to clients within IORs. The mechanism of propagation is
82
defined in the Quality of Service framework of the CORBA Messaging specification.
Figure 29 and Figure 30 show how the Client and Server Priority Models work.
Transforms are user-defined priority transforms that modify the CORBA Priority
associated with an invocation. The transforms take place when the invocation is
processed by a server. Priority transforms map Real-Time CORBA priority values to
other application-defined CORBA priority values. These can be used to implement
specific priority protocols. As PriorityTransform is an IDL native type, language
mappings are provided for different programming languages (C, C++, Ada and Java)
in the Real-Time CORBA specification. There are two types of priority transforms:
inbound and outbound. Inbound transforms take place when an invocation is received
by a server but before it is processed by a servant. Outbound transforms are applied to
ongoing requests from a servant to other CORBA objects.
Invocation executed at
priority 100 in the server
Client running at priority 100
Servant
Client
Stub
Skeleton
Client’s priority propagated
along the path to server in a
service context
ORB
POA
ORB
Figure 29: Client propagated priority model
Invocation executed at
priority 3347in the server
Client running at priority 100
Servant
Client
Stub
Skeleton
Client’s priority IS NOT
propagated along the path to
server in a service context
ORB
POA
ORB
Figure 30: Server declared priority model
83
Priority transforms are useful because there are situations in which the server or client
declared priority models are not enough. The advantage of priority transforms is that
they allow the application of the priority of execution taking external factors into
account. For instance, the occurrence of a critical external event or the workload of the
server computer could be used to decide on the priority of the request.
4.6.7
Synchronization
Synchronization is the satisfaction of restrictions that exist in the interleaving of
actions of different processes or threads [Burns 97]. One of the most difficult parts of
building a concurrent application is dealing with process or thread interaction. Threads
and processes are not independent of each other and system’s behavior depends on
their synchronization and communication. For our purposes, communication can be
understood as the passing of information between different processes or threads. This
can be achieved by means of shared variables or message passing.
Real-Time CORBA provides a mutex interface to allow threads execute in regions
protected by mutex objects. Objects of mutex type provide the degree of
synchronization needed to protect a critical section (this is known as mutual exclusion
and the MutEx word reflects that name). Real-Time CORBA provides two operations
in the RTORB interface to create and destroy mutexes, so the same mechanisms than
the broker uses for synchronization can be used by the application on top of the ORB.
This helps to reduce priority inversion as all the mutexes will use the same protocol
(e.g. a priority ceiling protocol).
4.6.8
Handling Concurrency
In the real world things happen in parallel. The term concurrency refers to an
expression of the parallelism present in the world. In the computer, parallelism is
achieved by concurrent programming. The term “concurrent” does not mean parallel
but “potentially parallel”. This is because concurrent programs or applications are
formed by a set of sequential processes that are (logically) executed in parallel
[Dijkstra 68]. Parallelism depends on how the collection of processes is executed.
•
•
•
The execution is multiplexed in one processor.
The execution is multiplexed in a multi-processor system where memory is
shared (parallel computing).
Execution is multiplexed in several processors and no memory is shared
(distributed system).
84
Concurrency can be expressed either at the level of processes (programs) or inside the
program (threads). In the latter case memory is shared by the different threads of the
program. Multi-threading is usually employed in addition to providing virtual
parallelism to allow the application increase its control over how different types of
services are executed (e.g. high-priority versus low-priority services) and to support
thread preemption to avoid priority inversion during lengthy request invocations. OS
APIs to create threads are not always standard (it is difficult to find full
implementations of the POSIX threads interface in all platforms) and a model of
thread interface (a wrapper class) is useful when developing for several platforms.
This is what Real-Time CORBA defines, a thread and a grouping of threads model
that allows heterogeneous distributed applications to be multithreaded in a simple,
uniform manner.
Real-Time CORBA introduces the threadpool concept (the grouping of threads) which
in a literal and straightforward definition is understood as a pool of threads. There are
two possible styles for the threadpool; with and without lanes (Figure 31). A lane is a
subset of the threadpool in which all the threads have the same RTCORBA::Priority
value. Different lanes in the threadpool may have different priority values.
Lane A
Lane B
Threadpool
Threadpool
Figure 31: Threadpool models, with and without lanes
The operations for the creation of threadpools allow to configure the stack size and
request buffering for the threadpool. The stack size is important because operation
arguments are stored in the thread stack and enough memory to avoid access
violations must be allocated for this purpose. As with other policy types, a threadpool
can be associated with several POAs at creation time (Figure 32) by using a
ThreadpoolId (an identifier assigned by the ORB to every threadpool). Other
interesting features can be set in the threadpool creation operations.
•
Static threads: If lanes are not being used, it is the number of threads to be
assigned to the threadpool. In a threadpool-with-lanes style it designates the
number of threads in each lane.
85
•
•
Dynamic threads: It is the number of threads that can be created dynamically
and that are allocated either to the threadpool or to the lane.
Priority: Without lanes it is the CORBA priority assigned to the static threads
in the threadpool. In this case, dynamic threads can be created at the priority
required to handle the invocation they were created for. If lanes are being
used, the CORBA priority is assigned to all the threads (statically and
dynamically allocated) in the lane.
In the styles with lanes, borrowing of threads from lower priority lanes can also be
specified. Once the borrowed thread has finished the request it is returned to the
original lane.
POA A
POA B
POA A
POA B
Threadpool A Threadpool B
threadpool
ORB
ORB
Figure 32: Assignment of threadpools to real-time POAs
4.6.9
Handling of Connections
Predictability is a main issue in a real-time system. In a conventional ORB, binding
does not occur until the first invocation on an object is made. In CORBA this is
known as implicit binding. In an implicit binding approach, communication resources
along the request path are established on demand, they are requested when they are
needed. The binding is a source of overhead that puts a penalty on the first invocation
made by a client. Real-Time CORBA anticipates this problem by making allowance
for explicit binding. Client applications can control the time when the binding on an
unbounded object reference is made by means of the validate_connection operation.
As with all models, implicit and explicit binding have their advantages and limitations.
Implicit binding helps the developer to conserve system resources and to share
connections among different clients or execution threads. However, this model is not
sufficient for real-time and control applications and a better way of managing
resources is needed. This is what the explicit binding model provides. In the explicit
86
binding model, connection to objects can either be deferred until the execution of the
request (as in the implicit binding) or resources can be reserved in advance to requests
(pre-establishing connections to server objects). This reduces latency and jitter for the
first invocation. Further, it is possible to use private connections or to assign a range
of priorities to a connection reducing also the possibility of priority inversion that
occurs in a multiplexed connection approach.
To support the explicit binding model, Real-Time CORBA provides the Priority
Banded Connections feature. A Priority Banded Connection is a connection with an
assigned set of PriorityBands. Priority bands assigned to a connection cannot be
reconfigured during the lifetime of the connection and no priority may be covered
more than once. As Figure 33 shows, it is also foreseen that non-contiguous ranges
can be formed and that it is not necessary to cover all CORBA priorities.
Banded Connections:
Invocation priority
determines connection
to use
client client
client
server
client
0 - 1300
ORB
ORB
9786 - 21340
Figure 33: Priority bands and explicit binding
The idea of banded connections is to allow clients communicate with servers using
multiple connections reserved for invocations made at different CORBA priorities. If
the priority of invocations is not respected by the transport it will become a source of
priority inversion.
The connection is chosen depending on the target object priority model. In the case of
the Client Propagated priority model, the band is chosen using the priority specified by
the client. If the Server Declared priority model is being used, its priority is published
in the IOR and its value is used to select the band.
87
Banded connections are configured by clients and the policies are applied to the client
side only. By default, the ORB provides a multiplexed connection for client / server
connections (Figure 34). However, it is possible to request a private transport
connection (Figure 34) by means of the PrivateConnectionPolicy object. The
advantage of private connections is that it ensures that no clients will interfere with
other requests in the case the reply from the server is needed. Reserving a private
connection allows to maintain the communications channel free of traffic while the
client is waiting for the reply to the request.
client client
Multiplexed:
client
Resource contention
by shared connection
client
ORB
ORB
client client
Private connection:
Exclusive use of a
connection by a client
client
client
ORB
server
server
ORB
Figure 34: Multiplexed and private connections in Real-Time CORBA
4.6.10
Invocation Timeout
Bounding the time in which a reply from a server must be obtained is a useful tool for
predictable system development. Predictability can be improved if system developers
know the upper bound of the time an invocation will be blocked waiting for the server
to answer. This functionality is achieved setting timeouts. Real-Time CORBA uses the
Messaging::RelativeRoundTripTimeoutPolicy to set the timeout for the reception of a
reply to an invocation.
In the case of Real-Time CORBA the timeout policy is a client-side only policy. It is
applied to clients who will cancel the request if the reply has not arrived before the
88
timeout expires. If the reply arrives at the client after the timeout has expired it is
simply discarded by the ORB.
4.6.11
Protocol Configuration
Real-Time CORBA applications may find that best-effort QoS requirements do not
meet their needs. In the general CORBA case, applications need not worry about the
location transparency in what refers to the protocol used for communications.
Anyhow, real-time systems like control systems need a tighter command and selection
of the protocols used for communication in order to improve end-to-end predictability.
Real-Time CORBA specifies a programming interface to select and configure the
underlying communication protocol (Figure 35). In CORBA, IOP instances (the
object that represents the interoperability protocol) contain an ORB protocol and a
mapping to an underlying transport protocol. Notice that two protocols are used, the
ORB message protocol (GIOP in the CORBA standard) and a transport protocol
(TCP/IP as prescribed by CORBA). The combination of both protocols results in the
Internet Inter-ORB Protocol of CORBA or IIOP, the standard CORBA
interoperability protocol. To provide control on the protocols used by the application,
Real-Time CORBA defines an interface that allows applications to control protocol
properties. It is possible to configure protocol properties either at the client-side or at
the server-side of an RT ORB.
Object Reference
Servant
Client
Skeleton
Stub
ORB A
ORB B
RTOS A
RTOS B
Invocation
ATM
TCP/IP
Figure 35:
CORBA
TCP/IP
Protocol
configuration
89
in
ATM
Other
Real-time
Protocols are specified by means of a protocol list which can be applied to POAs. The
list indicates the protocols supported by a certain POA and the order of the protocols
in the protocol list indicates the order of preference for the usage of protocols. The
policy is client-exposed, meaning that it is encapsulated in IORs to be consulted by
clients making invocations.
On the client-side, Real-Time CORBA defines a similar interface for the client
protocol policy. The difference is that the policy is applied at the time of binding to an
object reference. In the server side it is applied at the time of POA creation and the
policy is propagated from server to client in the IORs.
4.6.12
Real-Time Scheduling Service
The real-time scheduling service is a tool that enforces a certain scheduling policy to
be used across the whole real-time CORBA system. It is useful because specifying the
appropriate configuration parameters in all parts of the real-time system may be a
complex task. The scheduling service provides a form of central repository from
where a uniform scheduling policy for the whole system can be obtained by CORBA
objects.
4.7 Dynamic-Scheduling Real-Time CORBA
In order to generalize the Real-Time CORBA specification and meet the requirements
of a greater field of applications in real-time computing, another specification
(dynamic scheduling, see [OMG 01b] and [OMG 05]) has been created. The RealTime CORBA 2.0 specification tries to address static and dynamic distributed
systems. Dynamic distributed systems are those in which the processing workload of
the system is not well known before-hand or no bounds can be put to it so it is not
possible to perform an off-line analysis of the system. The specification generalizes
the concepts of distributed system scheduling and that of a distributable thread in
order to allow the application or the ORB have control on the following features.
• Any scheduling discipline may be employed.
• The scheduling parameter elements associated with the chosen discipline may
be changed at any time during execution.
• The schedulable entity is a distributable thread that may span node
boundaries, carrying its scheduling context among scheduler instances on
those nodes.
In order to provide more control over the temporal behavior of the application, it
interacts with the scheduler than can use one or more scheduling disciplines such as
90
Fixed Priority Scheduling, Earliest Deadline First, Least Laxity First, Maximize
Accrued Utility, etc. [Liu 00], [Burns 97]. The specification provides IDL interfaces
for all the above scheduling disciplines but it is possible to establish any other
scheduling discipline. The goal of the scheduler is to determine how best to meet the
schedule given a predicted use of system resources by the application in a certain
instant of time. The specification provides a framework in the form of IDL interfaces
that allows the development of portable schedulers.
4.7.1
Distributable Thread
In dynamic Real-Time CORBA, the notion of activity from Real-Time CORBA 1.0
has been replaced by that of distributable thread. For dynamic systems and in order to
achieve end-to-end timeliness a trans-node application behavior must be enforced.
This can be done by using the time and resources related parameters in a consistent
system-wide manner for allocation of resources. A trans-node application behavior
abstraction is defined (the distributable thread) for this purpose. A distributable thread
is a programming model abstraction. It is a thread that can execute operations in
objects without regard for the physical node boundaries (Figure 36). The distributable
thread is the schedulable entity in this specification.
Control flow
Object A
Object B
Object C
Figure 36: Control flow in a distributed processing
system
Each distributable thread may have several execution parameters (e.g. priority,
deadlines, utility functions, etc.) to specify the end-to-end timeliness in order to
complete the set of sequential operations in objects that may reside in different
physical nodes. The distributable thread transports scheduling information across the
distributed system.
91
Distributable Thread Forking
Forking is the part of a concurrency model that deals with the creation of a new
execution context. The specification allows for explicit forking of a distributable
thread by the use of the spawn operation. An example of forking is that of one-way
invocations as the distributable thread making the invocation is not blocked waiting
for the servant to process the invocation.
Scheduling Segments
A distributable thread consists of one or more scheduling segments. A scheduling
segment represents a sequence of control flow related to a set of scheduling
parameters. A scheduling segment has only one starting point and one ending point
which may span processor boundaries.
Distributable Thread Traversing CORBA Objects
Segment Scopes
BSS W
Seg. W
BSS X
Seg. X
ESS W
BSS Z
ESS X
Seg. Z
ESS Z
Object A
Object B
BSS- Begin Scheduling Segment
USS- Update Scheduling Segment
ESS- End Scheduling Segment
Object C
Portable interceptor
Application call
Figure 37: Distributable thread with nested scheduling segments
Figure 37 shows an illustration of a distributable thread with nested scheduling
segments. Where the segment X begins, the scheduling context of segment W is put
aside and segment’s X scheduling parameter elements are used for the distributable
thread. When segment X ends, the scheduling parameters for segment W are used
until it ends.
92
4.7.2
Real-time CORBA Scheduler
The Scheduler is an extension to Real-Time CORBA that manages the scheduling
requirements and parameters of the applications that run on top of the broker. The
scheduler decides the order of execution of the applications on the distributed nodes of
the CORBA system. To decide on the execution eligibility in the CORBA systems the
scheduler is based on the following characteristics.
• The scheduler responds to application requests (to define scheduler elements)
and in response to application actions (e.g. such as invocations by using the
Portable Interceptor interfaces).
• The scheduler uses the information provided by the application to decide on
the eligibility of threads.
• The scheduler architecture is based upon the concept that the distributed
system can be considered as a set of distributable threads.
• It is supposed that the schedulability of the system can be addressed by
managing the allocation of resources to distributable threads.
• The distributable threads and the scheduler interact at specific scheduling
points such as in transitions to new processors where scheduling information
must be re-interpreted.
• The scheduler is a pluggable scheduler. If an ORB has a scheduler installed,
all applications that run on that ORB use that scheduler.
Scheduling Points
The scheduling points are the points in time and/or in code where the scheduler is run.
This may result in a change of the current schedule. The defined scheduling points are
shown below.
• Creation of a distributable thread.
• Termination or completion of a distributable thread.
• Beginning of a scheduling segment.
• Update of a scheduling segment.
• End of a scheduling segment.
• A CORBA operation invocation, specifically the request and reply
interception points provided in the Portable Interceptor specification.
• Creation of a resource manager.
• Blocking on a request for a resource.
• Unblocking as result of the release of a resource.
93
Schedule-aware Resource
The specification allows the creation of schedule-aware resources via a Resource
Manager. The resources can have scheduling information associated with them.
4.7.3
Advice about Real-Time CORBA
Originally, CORBA was not planned to be deployed in hard real-time systems.
Therefore several sources of non-determinism can be identified that could cause the
system to miss its deadline: marshalling of parameters at the client side, protocol
queues of the client, delays on the transport media, protocol queues on the server,
dispatching of threads and requests, and un-marshalling of parameters at the server
side.
Nevertheless, there is no clear way of reducing complexity by subdividing a system
into subsystems and retaining the temporal behavior. Thus, the implementer is left
with the complexity of the whole system. Introducing composability while preserving
end-to-end predictability is still an open issue. Especially analysis of a RT-System
according to the real-time CORBA specifications is complicated by features like
borrowing threads among threadpools.
4.8 Minimum CORBA
Minimum CORBA [OMG 02k] defines a cut-down profile of CORBA, not of Realtime CORBA, for those applications with limited resources. Regarding conformance
to the full CORBA specification, Minimum CORBA follows these guidelines:
The Minimum CORBA specification supports all OMG IDL to allow
maximum compatibility between minimum and full CORBA applications.
This point is important as most Real-Time CORBA brokers are built around a
Minimum CORBA broker.
Minimum CORBA applications must be fully interoperable with CORBA
applications as Minimum CORBA applications may be part of a greater
system running CORBA applications.
Dynamic aspects of CORBA are omitted in Minimum CORBA as for the
target applications of Minimum CORBA many decisions are made at design
time.
94
4.9 Fault-Tolerant CORBA
There are various kinds of applications with a need for fault tolerance, beginning from
large critical systems (such as air traffic control systems) to smaller critical systems
(such as medical systems) and embedded applications (such as manufacturing control
applications). A standard that attempts to meet all of the requirements of this wide
spectrum of applications might satisfy many needs only poorly, or might be too
complex to implement. The Fault-tolerant CORBA specification [OMG 02b] therefore
represents a number of compromises in order to support most of these systems.
Fault tolerance depends on entity redundancy, fault detection, and recovery. The entity
redundancy by which this specification provides fault tolerance is the replication of
objects. This strategy allows greater flexibility in configuration management of the
number of replicas, and of their assignment to different hosts, compared to server
replication. Replicated objects can invoke the methods of other replicated objects
without regard to the physical location of those objects.
The standard supports a range of fault tolerance strategies, including request retry,
redirection to an alternative server, passive (primary/backup) replication, and active
replication which provides more rapid recovery from faults. Applications that require
the Fault Tolerance Infrastructure in order to control the creation of the application
object replicas are supported as well as applications that control directly the creation
of their own object replicas and applications that require the Fault Tolerance
Infrastructure to maintain Strong Replica Consistency, both under normal conditions
and under fault conditions.
Support for fault detection, notification, and analysis for the object replicas is
supported. Thus, allowing the development of applications that require the Fault
Tolerance Infrastructure in order to provide automatic checkpointing, logging and
recovery from faults as well as applications that handle their own fault recovery.
The standard aims for minimal modifications to the existing application programs, and
for transparency to replication and to faults and, it defines minimal modifications to
existing ORBs that allow non-replicated clients to derive fault tolerance benefits when
they invoke replicated server objects.
4.9.1
Grouping Abstractions
In Fault-Tolerant CORBA there are two types of grouping abstractions; the object
groups and the fault tolerance domains.
95
Object Groups
In order to achieve CORBA-objects fault tolerance, several replicas of the object are
created and grouped in an Object Group. The object group can be addressed as a
whole by the use of an Interoperable Object Group Reference (IOGR) which is
exported by the server to be used by CORBA clients (client-exposed in OMG
terminology). The IOGR is created by the Replication Manager (see next sections).
The clients invoke their requests on the object group and are processed by the object
members of the group. Each individual replica keeps its own IOR but the abstraction
of the object group provides two advantages from the client point of view.
•
•
Replication Transparency: The client objects are not aware that the server
objects are replicated.
Failure Transparency: The client objects are not aware of fault in the server
replicas or of fault recovery.
Fault Tolerant Domains
FT CORBA also deals with large applications that have a need for fault tolerance.
Such applications manage thousands of objects and span several locations so it is
inappropriate to consider them as a single entity. To address this problem, fault
tolerant CORBA defines the concept of fault tolerance domains (Figure 38). A fault
tolerance domain usually contains several hosts and many object groups being
possible for a single host to support several fault tolerance domains.
TUVienna
Location
Host 3
A1
Host 1
IIOP
Message
over TCP/IP
ORB
without
support for
Fault Tolerance
B2
E2 F1
C1
Host 7
Host 2
Gate
way B1
Host 5
C3
D1
C2
Host 6
E3 F2
Host 4
SCILabs
Domain
E1
Wide Area
Domain
UPM
Domain
Figure 38: Fault tolerance domains
All the objects and groups in a fault tolerance domain are managed by a single
Replication Manager. Fault tolerance domains do not isolate objects in one domain
from those of others. Objects in a domain can invoke to and be invoked by objects in
96
other domains. Figure 38 shows an example of fault tolerance domains which are
filled in blue in the figure. Hosts are shown in green and members of an object group
are shown in dark blue. The members of object group B are denoted as B1, B2. The
same notation is valid for groups A, D, C, E and F.
The fault tolerance domains allow applications to be arbitrary in size, putting no limit
to application scale. This is achieved by letting replications managers handle a smaller
number of objects than that of the whole system. Fault tolerance properties can be
assigned either to object groups or to fault tolerance domains. Number of replicas or
replication style (passive, warm passive or active) and other properties can be applied
to all the objects of a group, domain or to all the object groups of a specific type.
4.9.2
Architectural Overview
A fault tolerant CORBA system needs the infrastructure shown in Figure 39. A
Replication Manager, Fault Notifier and a Fault Detector object are implemented as
CORBA objects. Logically, only a single instance of these objects exists in a fault
tolerance domain but they are physically replicated for fault protection as all the other
objects of the application. The Replication Manager of Figure 39 inherits the Property
Manager, Object Group Manager and Generic Factory interfaces.
create_
object()
set_
properties()
Replication
Manager
notifications
Fault
Notifier
create_
object()
Fault
Detector
is_alive()
fault reports
H1
H2
H3
Client
Server
C
CORBA
Server
S1
ORB
Logging
Mechanism
S2
Factory
Fault
Detector
Factory
Fault
Detector
CORBA
ORB
CORBA
ORB
Recovery
Mechanism
Logging
Mechanism
Recovery
Mechanism
Logging
Mechanism
Figure 39: architectural overview of a fault tolerant CORBA system
97
The PropertyManager interface allows users to specify fault tolerance properties of
object groups. Replication management is controlled by the use of the GenericFactory
and the ObjectGroupManager interfaces. The GenericFactory interface is able to
create replicated objects on application demand. The GenericFactory is not used
directly; it is the Replication Manager who will invoke the factories on the hosts
where the replica is to be created. The ObjectGroupManager operations are designed
to add, remove or control the location of members of an object group. The figure
shows three hosts H1, H2 and H3. The client C on H1 is invoking a replicated server
with two replicas S1 on host H2 and S2 on host H3. The Factory and Fault detector
objects in each host are not replicated as occurs with the service objects on top of the
figure.
All the application objects inherit a PullMonitorable interface that a Fault Detector
invokes. It is a kind of watchdog invoking an is_alive() operation. Faults detected by
in-host Fault Detectors are notified to the Fault Notifier which passes the notifications
to the Replication Manager.
In the case that the passive or warm passive replication styles are used, only one
member of an object group executes the requests and sends the replies. On a faulty
condition, the Replication Manager can restart the primary member of the object
group or can promote a backup member to primary member.
In the case of active replication all the members of an object group execute
invocations independently and in the same order so as to keep the same state. When a
fault occurs in one member the application continues with the results of other member
without waiting for fault detection and recovery. There is a message handling
mechanism that detects and suppresses all replies except one which is sent to the
client.
4.10 Data Distribution
Many real-time applications have a requirement to model some of their
communication patterns as a pure data-centric exchange, where applications publish
data which is then available to the remote applications that are interested in it. The
Real-Time Data Distribution Service [OMG 03e] specification targets at solving the
following problems:
• Predictable distribution of data with minimal overhead since it is not feasible
to infinitely extend the needed resources.
98
•
Scalability to hundreds or thousands of publishers and subscribers in a robust
manner. Data-centric communications decouples senders from receivers; the
less coupled the publishers and the subscribers are, the easier these extensions
for scalability become.
The Data-Centric Publish-Subscribe (DCPS) model has become popular in many realtime applications. Compared to distributed shared memory which is a classic model
that provides data-centric exchanges, this model is easier to implement efficiently over
a network and allows the required scalability and flexibility. It builds on the concept
of a “global data space” that is accessible to all interested applications. Applications
that want to contribute information to this data space declare their intent to become
“Publishers” and applications that want to access portions of this data space declare
their intent to become “Subscribers”. Each time a Publisher posts new data into this
“global data space”, the middleware propagates the information to all interested
Subscribers.
Another common need is a Data Local Reconstruction Layer (DLRL) that
automatically reconstructs the data locally from the updates and allows the application
to access the data as if it were local. In that case, the middleware not only propagates
the information to all interested subscribers but also updates a local copy of the
information.
4.11 CORBA Component Model
Components are a way to increase productivity, reuse existing code and reduce
complexity. The improvement of all those factors shortens the time to market of any
product. Hence, the interest in the development of a component model for CORBA.
Developing applications or systems with components can be thought of as assembling
the application from pieces, with little development involved. Instead of engineering
components, the focus can be put on the problem to solve. As the components are
reused over and over they refine their quality, producing by composition systems of
better quality.
In the distributed object model of CORBA, the problems of heterogeneity, portability
and interoperability are resolved in an elegant manner. All those problems that
CORBA resolves can be summarized in one word: integration. CORBA is about
integration. However, some problems like those explained in Section 3.6.3 prevent in
some degree the development of a component market for CORBA. The CORBA
Component Model (CCM) [OMG 02l] builds on the features of the CORBA
distributed object model to deploy true components.
99
The distributed component model of CORBA defines an architecture for component
definition and interaction, either from the client-side or the server-side (business
components to build application servers). One of the critics of the CORBA object
model was the lack of packaging and deployment facilities. In the CCM, a packaging
and deployment model is defined to deploy binary multi-lingual executables. One of
the complexities of CORBA was dealing with non-functional properties of objects
(e.g. lifecycle of objects, persistence, etc.), in the CCM, a framework is defined in
which non-functional attributes of components can be specified. CCM solves these
problems without forgetting about being vendor-independent or multi-language or
multi-ORB interoperable. The CCM specification deals with several aspects of
CORBA components.
•
•
•
•
•
•
4.11.1
The CCM presents an abstract component model where components are
represented as extensions to objects by extensions in the IDL and in the object
model.
The CCM defines a Component Implementation Framework which defines a
Component Implementation Definition Language (CIDL).
The CCM defines a Component Container Programming Model. The
programming model is useful for the component implementer and for the
client programmer point of view. The Container model integrates with nonfunctional features of components such as events, transactions, persistence and
security.
The CCM defines packaging and deployment specifications for components.
It defines an interoperability specification with Sun’s EJB 1.1.
It defines a MOF-compliant component meta-model to express the extensions
to IDL of the CORBA Component Model and another one for the Component
Implementation Framework.
What is a CORBA Component?
The Component is a basic CORBA meta-type that extends the functionality of the
Object meta-type. As such, they can be expressed in IDL and represented in the
Interface Repository. Components are represented by an object reference and its
definition is made in an extended version of IDL.
Components present surface features that allow them to interact with other elements
of the application or clients. The surface features are called ports and exist in various
forms.
100
•
•
•
•
•
Facets. They are distinct named interfaces provided by the component for
client interaction (services provided by the component).
Receptacles. Which are named connection points that describe the
component’s ability to use a reference supplied by some external agent
(services requested by the component).
Event Sources. Which are named connection points that emit events of a
specified type to one or more interested event consumers, or to an event
channel.
Event Sinks. Which are named connection points into which events of a
specified type may be pushed.
Attributes. Attributes are named values intended for component
configuration.
In the CCM there are two types of components, basic and extended. A basic
component can only have attributes whereas an extended component can have any
type of port (Figure 40).
Component interface
Receptacles
OFFERED
CORBA
Component
Event
sinks
Event
sources
REQUIRED
Facets
Attributes
Figure 40: Extended CORBA Component and ports
4.11.2
Limitations of the CCM for Distributed Control
The CCM provides a standard way to deploy distributed systems with a great degree
of software reuse and without the need of very skilled staff. While this is true for
business-like systems, the construction of distributed control systems in this fashion is
still a field of research. Today’s Real-Time CORBA ORBs can be used in a variety of
101
real-time systems with good results. This is the result of several years of development
and improvement of the ORB QoS features needed in this type of domains. The same
holds for a component approach to system development for control purposes. The
CCM builds on top of CORBA which can be enabled to handle the requirements of
control or real-time systems. Nevertheless, the CCM needs improvements to able to
communicate these requirements to the underlying ORB and this is not dealt with in
the CCM specification. Among the problems to be solved in order to use the CCM in
distributed control systems there are the following.
•
•
•
The CCM has no mechanisms to enforce real-time policies. The CCM needs
to able to configure the QoS control mechanisms of Real-Time CORBA.
The CCM introduces new classes and types atop of CORBA. This results in
more code to execute, introducing overhead in the time and memory needed
by the component to perform the service. This is a problem in systems with
tight requirements or scarce resources.
The CCM is bulky and has a big memory impact. The Lightweight CCM
specification [OMG 04c] tries to mitigate this problem.
One of the advantages of the CCM is that it is able to deal with non-functional
properties of the components as the events or transactions. A possibility for real-time
systems is to extend the CCM to deal with the non-functional properties of the
component’s real-time requirements.
4.11.3
The COMPonent Approach
Embedded (COMPARE) Project
for
Real-time
and
The COMPARE project (IST-004669) is funded by the IST Sixth Framework
Programme. The project consortium is composed by the following partners: Thales
Communications, Schneider Electric, PrismTech, Trialog, CEA-LIST and UPM.
COMPARE recognizes the limitations already presented of the component model of
CORBA to be used in real-time and embedded systems and is focused at the
specification and implementation of a component framework that allows dealing with
the complexity and performance issues of this type of systems.
COMPARE starting point is that complexity is already addressed in other types of
systems (mainly business systems) by the use of component frameworks (e.g. SUN’s
EJB). COMPARE tries to apply this same idea to the area of real-time and embedded
systems by tailoring and extending the Lightweight CORBA Component Model. This
means studying the LwCCM to make the necessary modifications. Concretely, the
following items are addressed (see Figure 41).
102
•
•
•
•
Interaction models for the adaptation of the LwCCM to the RT/E area.
Current supported interaction models are synchronous call of methods and
asynchronous sending of events (messages). For RT/E systems, other models
such as streaming ports are worth to study. In any case, Quality of Service
should be supported for any of these interaction kinds.
Limitations of LwCCM Containers. The currently supported services are not
suitable for RT/E and work is needed in areas regarding real-time properties
(e.g. scheduling support), proper activation of components (e.g. periodic
activation), etc.
Assembly, configuration and deployment are well defined in the classical
CCM. However, they have to be adapted to specific constraints due to
embedability.
The underlying execution infrastructure (RT CORBA or others) must be
adapted to the constraints of the targets.
In Section 5.13 of this Thesis some extensions to IDL for the handling of periodic,
aperiodic and sporadic tasks are presented in a preliminary proposal for the future
work to be carried in COMPARE.
Predefined components
For RT/E
Interaction models for RT/E
component
Containers for RT/E
Assembly, configuration,
deployment, supervision
for RT/E
Preexisting
component
Preexisting
component
container
container
container
Execution
Execution
Platform
Infrastructure
Services for RT/E
Figure 41: Areas of work for the RT/E component framework
103
Chapter 5
Using CORBA in Control Systems
105
5.1 CORBA Weaknesses
Making distributed control is not a simple task. In the bold ICa endeavor to facilitate
the construction of such systems, CORBA has been selected as the basic infrastructure
middleware to integrate control systems at all levels of the plant (operational, tactical,
strategic). But before the technology is deployed the question arises whether CORBA
can address the control requirements at all these levels.
During this investigation it has been found out that CORBA proves itself worthy in the
upper levels of the control pyramid or stack of layers. There are already other
technologies capable enough to provide support for applications at these levels but still
CORBA provides better support for integration than the rest. The real problems with
CORBA begin to show up as we get closer to the lower simple basic loops of control.
This is one focus of this Thesis in which extensions to the CORBA model are used to
further span its field of application.
Autom
ation
Appli
catio
ns
Infras
tructu
re
Basic
S
Islan
ds o
f
ervic
es
Auto
m
ation
Figure 42: A typical automation infrastructure
In Figure 42, the typical example of a plant automation layering is shown. In the
example, the lower layer is cluttered with what have been called islands of
automation. At the beginning of plant automation, islands operate independently of
each other frequently using different core technologies to carry out control. It is not a
107
bad situation if all systems in the plant are independent of each other. In the real
world, the enterprise can be seen as a whole and no part of it or individual department
stands alone without the others. It is just like a composition association of UML
entities; their life cycles are tied to each others’.
The islands of automation are implemented in most cases by proprietary technology or
automation solutions. In order to communicate and to get a common understanding of
the enterprise automation as a whole, basic automation services are needed, which are
built upon the islands of automation. This basic services layer is where process
visualization, alarm and event management, security, device integration (OPC for
instance) or messaging services are implemented. As the requirements for this layer
are not as stringent as in the islands of automation layer, there is a bunch of
technologies that can be used at this level. Until this point, there is not a unified view
of the plant. There are only services that allow to interact and to examine the behavior
of the different islands of automation. Today’s modern plant automation solutions
deploy an infrastructure upon the basic services that allow plant-wide automation
applications to be built. This is shown in the last two topmost layers of the figure. Also
at this level, specialized CORBA interfaces can be used to provide an infrastructure
for plant-wide automation. CORBA integrates horizontally in each layer and vertically
in a cross-layer flow of information. As an example of a top automation layer
application, the reader might think of Manufacturing Execution Systems.
So CORBA is a good fit in all the upper layers of plant automation. In fact, it fits
better than other systems because when information crosses layers upwards or
downwards, different protocols are tipically used. The lower layer will use some field
bus approach to provide the basic level of automation. In the basic services layer the
protocol is usually changed to something more mundane as OPC which is used to a
great extent. And from here to the upper layers other protocol conversions may be
used depending on the types of applications deployed in the plant. Protocols vary from
Suite Link to MOM proprietary protocols.
The focus here must be put on the lower layer, where resources are low and the
requirements for the exchange of information are stringent. By the use of CORBA in
all the upper layers of the figure, heterogeneity is resolved by means of CORBA
integration. The objective is to integrate also the lowest layer. This layer is always
very costly in economic terms due to the use of vendor proprietary protocols and
specialized hardware. By integrating CORBA into this layer process information and
systems command and control can be made available plant-wide (with the due security
authorizations).
108
The case of islands of automation is just an example scenario where CORBA can be
used in distributed control. There are other types of distributed systems in which there
is not such a hierarchy of layers but in which CORBA can bring numerous
development advantages. It is the field of embedded distributed control systems.
Examples of this type of systems are brake-by-wire systems, ABS or cabin pressure
control systems in aircrafts. In all these systems, distributed specialized hardware and
software are used to carry out the control logic and to communicate the devices in a
network with hard deadlines.
In order to develop core assets for control systems infrastructure to build upon, it is
needed that the weaknesses of CORBA regarding control in hard real-time and
embedded device conditions be solved. In this case, the infrastructure is a CORBA
ORB. Although CORBA was not initially foreseen for industrial systems, it has
specifications for real-time, fault-tolerance and embedded systems. Unfortunately,
these specifications, specially the real-time specification, miss to treat some of the
most important points for control systems. To overcome these difficulties or
weaknesses, a completely revamped CORBA ORB has been developed to incorporate
functionality for control purposes. Specifically, the following issues in the control
domain have been addressed in this Thesis work.
•
•
•
•
•
•
•
•
Sources of overhead or non-determinism in the ORB.
Communication models in control systems and in CORBA.
CORBA control loops.
Event-triggered systems versus time-triggered systems.
The notion of time in control systems.
Time-stamping.
Protocol statistics.
Additionally, this Thesis shows the need to address periodic, aperiodic and
sporadic activities.
These and other issues are being continued within the IST project COMPARE in the
context of embedded components. With the items considered, it is possible to have an
infrastructure upon which core assets for control systems can be built. It is not
possible to have assets with real-time or other control properties if the underlying
system, the middleware, does not support them. In this chapter, the modification of an
ORB to support these attributes is introduced. Means for deploying CORBA in the
lower layers of the automation infrastructure and in embedded systems are presented.
Extensions to and profiling of real-time CORBA and other interfaces are introduced
that allow building applications in small close-to-process devices.
109
5.2 CORBA Control Systems Domains
The collection of domains to be considered in the scope of CORBA Control Systems
is the following:
• Distributed systems (basically the CORBA Domain): The main characteristic
of this type of systems is the distribution of functionality in a collection of
components that are running (or can be running) in a collection of processors.
CORBA focuses specifically in distributed object systems where the
interaction semantics is based on object method invocations.
• Distributed real-time systems (the Real-time CORBA Domain): The
CORBA base specification does not address topics of real-time systems. The
CORBA approach is a best-effort approach where the application developers
do not have control on the resources usage to guarantee a real-time behaviour.
RT-CORBA addresses these issues providing mechanisms for resource
control (processor, communications, memory, etc).
• Distributed reliable systems (the Fault Tolerant CORBA Domain): The same
can be said of CORBA for development of robust systems. Fault tolerance is
not addressed in the basic CORBA specification. The standard for Fault
Tolerant CORBA aims to provide robust support for applications that require
a high level of reliability, including applications that require more reliability
than can be provided by a single backup server. The standard requires that
there shall be no single point of failure.
• Embedded distributed systems (the Minimum CORBA Domain): For some
applications CORBA is too large to meet exacting size and performance
requirements. Such scenarios require a cut-down version of CORBA.
• Control Systems: Control systems are a domain on its own right. The issues
concerning the development of (distributed) control systems are a main
concern here.
5.3 The Integrated Control Architecture (ICa) ORB
The ICa ORB is the result of a long-term effort towards a solution for distributed
control systems that has been extensively funded by several European Union research
projects in the framework of the ESPRIT and IST programmes.
This ORB originated from the need of reducing the effort employed to develop
distributed communications among the processes and devices of distributed control
systems in the ICa model. In the late eighties, the obvious solution was to implement
110
an abstraction layer above the details of UDP or TCP/IP programming interfaces to
ease the implementation of communications. This was the first attempt to provide
some leverage to the problem of communication and gave birth to CCL, the Complex
Communication Layer library. This effort was carried out during the works related to
the CONEX and HINT projects.
In the final eighties and the start of nineties, the OMG began its effort towards an
Object Management Architecture (OMA) and its best known specification CORBA.
To our research group at UPM, it was clear that an enormous effort was being put in
the promotion of the OMA and that the idea of moving to the distributed object world
was a promising one. Work began in order to develop an ORB for the industry based
upon the ideas of the OMG and in the reutilization of the work done in the CCL
library. Basically, the work consisted in the construction of a layer above CCL that
allowed us to give system communications an object-oriented approach. An ADL
(Agent Definition Language) compiler that was able to generate stubs for interface
objects and some multithreading utilities for automatic concurrency support were also
developed. The DIXIT project was focused on the development of these tools and its
test in the petrochemical plant of Repsol in Tarragona. Although this first ORB was a
great advance over the former CCL library as it allowed expressing the system
architecture and its interfaces in agent form, it lacked most of the advantages of
CORBA.
•
•
•
•
It followed the philosophy of CORBA but was not fully compliant to the
CORBA specification. Thus, all ORB utilities were developed proprietarily.
Interoperability was not possible as no IIOP was built into ICa ORB.
Interoperability is the primary means to achieve integration and the lack of
IIOP precluded ICa from being widely deployed in large complex and
heterogeneous systems.
In a real-time distributed system, the transmission of quality of service (QoS)
properties in the system is essential. ICa did not support this type of nonfunctional information to be transmitted in requests.
Heterogeneity also means heterogeneous transports at the network level. ICa
did only support TCP/IP or UDP which are not deterministic transports. Also,
there was not support for connection management in order to enforce the
needed quality of service.
All the above circumstances were identified among others, and more funding was
obtained to demonstrate that it was possible to incorporate those features efficiently
into a distributed-object approach for control systems. Under the umbrella of the
DOTS project a completely new ICa ORB was developed, this time considering the
111
CORBA and the real-time CORBA specifications in an approach oriented towards the
control world. A full real-time CORBA ORB was developed and extended with
additional interfaces and, its fitness for distributed control systems was proved in the
scope of SAS (Substation Automation System). The feasibility of embedding the ORB
for control in Remote Terminal Units and the interoperability approach with other
remote systems running in different platforms and written in other programming
languages was demonstrated. This was a major step in ICa as the span of possible
applications opened enormously.
Although DOTS demonstrated that ICa can be used in traditional control systems, it
had not undergone the test of hard real-time. In this Thesis, the effort is put into
developing a technology capable of being employed in a span of control applications
as wide as possible so there was the need to provide support for such hard
requirements. In the case of hard real-time, the problem is the lack of determinism in
the ORB. With hard deadlines, a deterministic approach to an ORB is needed. This is
not addressed either by current CORBA or real-time CORBA specifications. A new
research project, HRTC or Hard Real-Time CORBA, was launched with the objective
of identifying the sources of non-determinism in the ORB and to integrate
functionality in ICa which allowed running the ORB over deterministic network
transports.
With these achievements it is possible to use a single ORB or a combination of ORBs
(depending of the layer requirements where the ORB runs) at all levels of the control
hierarchy and, to integrate heterogeneous systems and networks transparently.
5.4 Levels of Requirements
In a real plant or system there will be a varying collection of requirements depending
on how close specific parts of the system are to the process. This means that most
large real-time systems are multi-rate real-time systems (a form of a sampled data
system) where the sampling period for each variable may have different stringency
requirements. Multi-rate is just a consequence of the different process variable
dynamics and of the different sampling periods needed to achieve smooth input from
them.
Although multi-rate systems can be single-layer systems, it can be assumed that all
systems where there is a hierarchy of layers will have multi-rate sampling. In this case,
the different timeliness requirements of all layers are made self-evident.
112
Complex control is the typical case of a multi-rate layered system where output from
upper layers is the reference input of several controllers in the lower layers. Although
in some cases it is possible for operators to carry out control directly on the lower
level controllers (local control), they usually interact with upper layers high-level
interfaces. This situation matches the example shown in Figure 42.
Layers do not have to be physically linked to each other. There are systems where the
layers are linked over the air. Figure 43 shows an example of such a system.
Commands
Responses
Operator-system
Interface
-
From sensors
Air-traffic
Control
State Estimator
Virtual Plant
Navigation
Flight
Management
State Estimator
Virtual Plant
CORBA
-
-
Flight
Control
State Estimator
Air Data
Physical
Plant
Figure 43: CORBA air-traffic flight control system (ATC system
structure from [LIU 00])
In this CORBA-based air-traffic flight control system there are different timing
requirements for CORBA in the different layers of the hierarchy. The system works as
follows [Liu 00]. The air-traffic control upper layer manages the flights headed to each
destination airport. It develops its mission by assigning a sequence of known
geographical points and a time of arrival (metering fix) to every flight. This
information is the input for the on-board flight management system in the middle
113
layer. The flight management system calculates a time-dependent flight path to meet
the next metering fix at the specified arrival time. The flight management system
provides parameters as cruise speed, descend or ascend rates, turn radii, etc. as input to
the flight controller in the lower layer and closest to the physical plant.
It is easily understood that the periods of computation and response to stimuli in each
of these layers is different. The CORBA arrow turns deep red where timing
requirements are harder. For the upper layers of the figure, computation time for the
control law can be in the order of minutes while for the lowest layer the order of
milliseconds must be used. Remember that the lowest layer is a distributed system in
itself and that communication of information must be carried out.
In order to tackle the requirements of the lower layers of this type of systems, the
sources of non-determinism of ICa must be known. By knowing the sources of nondeterminism performance improvements can be achieved and the overall behavior of
the ICa ORB can be made more predictable.
5.5 Sources of Non - Determinism
ORB sources of non-determinism have been largely studied ([Gokhale 97], [Schmidt
98]). In order to build hard real-time distributed control systems based in CORBA,
there is a need to be able to account for the behavior of the ORB regarding the
following attributes.
•
•
•
•
•
Determinism and predictability.
End-to-end latency and speed.
Throughput.
Scalability.
Footprint.
The comportment of an ORB regarding these and other attributes defines its fitness for
the control system purpose. Tuning of a real-time ORB is not easy as it is composed of
very complex pieces of software that interact and, finding the origin or source for nondeterminism is not always and easy task. Broadly speaking, the source of applicationlevel non-determinism has four main causes.
•
Lack of Quality of Service interfaces. The CORBA standard does not
provide interfaces to control end-to-end QoS. Real-time CORBA partly
alleviates this problem by allowing the developer to have control over system
resources but still some interfaces are missing.
114
•
•
•
Quality of Service is not enforced by the ORB. Once a client application
has delivered the request to the ORB, it must guarantee that the desired QoS
for the request is used end-to-end. At the server-side ORB, the request must
be properly dispatched according to the QoS and the reply returned to the
originating ORB. This is not true for most ORBs.
Lack of language and platform support for real-time features. CORBA
ORBs are developed using programming languages and run on top of
operating systems. This means that although CORBA may specify interfaces
regarding end-to-end predictability the underlying infrastructure may not do
so. At the end, ad-hoc and non-optimal implementations of critical features
are incorporated to ORBs.
Lack of ORB performance optimizations. Behavior of the broker is heavily
dependant of its architecture and implementation. Thus, the parameters
identified at the beginning of the section can be improved by design and
implementation of efficient algorithms. This item is a continuous effort as it is
known that “software development never ends”.
The ICa ORB has been designed with these concepts in mind. In some cases, it is
feasible to improve behavior by design (performance optimizations) while in others
(language and OS support) it is clearly a matter of providing the best possible
implementation with the tools available. Regarding lack of ORB performance
optimizations, Figure 44 shows the points in the ORB’s general architecture where
improvement is usually needed.
Impact on ORB architecture of predictability issues
Servant
marshalling
marhalling
Client
Skeleton
Stub
request
dispatching
marshalling
marhalling
Memory mgmt
ORB
POA
thread
dispatching
delay
ORB
buffering
buffering
Memory mgmt
Figure 44: Points of predictability improvement
115
ICa ORB has been optimized to provide a consistent behavior in all the areas shown in
the previous figure. This information is presented in the next section. But
predictability and determinism are also improved by control of system resources.
System resources control is specified by real-time CORBA and ICa ORB provides an
implementation of this specification. Figure 45 shows the possibilities for control of
resources that have been implemented in ICa ORB and that conform to the real-time
CORBA specification.
Provide predictability by controlling ORB behavior
Servant
Servant
Client
Skeleton
thread
priority
concurrency
Stub
connection
POA
POA
protocol
ORB
ORB
Thread
thread
priority
Invoke
priority
Figure 45: Resource control features in ICa
Basically, it is possible to control the QoS at certain levels during the duration of an
invocation between client and server objects. This includes controlling the thread
priority of an invocation in the client application as well as the management of
connections and protocols configuration for the client side. At the server side, QoS is
respected and resources are scheduled in accordance with it. Section 5.6 presents the
general improvements for performance included in ICa.
5.6 General ICa ORB Performance Improvements
As Figure 44 shows, there are several items in a CORBA ORB where optimizations
must be worked out. ICa ORB has been built upon efficient implementations of
critical code and specialized algorithms have been used to solve involved problems of
the ORB. The following items have been addressed.
116
5.6.1
Client Stubs and Server Skeletons
In CORBA, the basic mechanism of communication between the application objects
and the ORB is resolved by the use of client stubs and server skeletons. Stubs and
skeletons are pieces of software that are automatically generated by IDL compilers.
CORBA objects are distributed, this means that the object can be used locally as if the
object really were in the same host than the client (in fact in modern operating systems
it must be in the same address space to obtain a direct request) although it may be
many miles away. This ability to make local requests to remote objects makes one of
the major advantages of distributed objects; systems programming is kind-of objectoriented.
To be able to request a remote object services in an object oriented way, the IDL
compiler builds a stub of the remote object. This stub is a shell with the appearance of
the remote object that interacts with the client application and the ORB to forward the
request to the real server object. At the server side, the ORB delivers the request to the
application-implemented object services by means of the server skeleton. The skeleton
maps the information received from the ORB to the signature of the operation
implementing the service.
All this flow of information through stubs and skeletons must be made very efficiently
so they must be optimized. In ICa ORB, built in C++, the stubs and skeletons take
advantage of the pointer concept to make a very fast implementation. Arguments
pointers are identified and paired to their marshalling function pointers and passed
onto the ORB in arrays of pointers to carry out further work. Server skeletons are
implemented in much the same way but they also support or deal with the burden of
helping resolve request dispatching as explained in the appropriate section below.
5.6.2
Marshalling
In a CORBA ORB, a network format for data is used to communicate objects that
reside in different machine architectures, operating systems or that have been written
in different programming languages. Marshalling and de-marshalling take place at the
client and server sides. As the process consists in the formatting to the proper
representation of the operation to be invoked, the arguments to be passed between
client and server, and some operation context information (including QoS information
in a real-time ORB), the algorithms employed in the process must be as efficient as
possible. The format used in CORBA is called GIOP or General Inter-ORB Protocol.
ICa ORB includes a library to format GIOP messages that has been written in an
efficient manner. Additionally, specifying an interface among the GIOP and the ORB
117
by means of a library allows code for GIOP to be changed in an easy way, being
possible to implement new marshalling strategies in a small amount of time.
5.6.3
Memory Management
In a predictable ORB, memory management is an important issue because it affects
how the system behaves. Memory is a system resource that must be controlled in order
to ensure proper system operation. In the case of embedded systems, memory is a
critical matter as the ORB must never go beyond the embedded platform capabilities.
The continuous use of dynamic memory (malloc or new way of requesting memory) is
expensive and the frequent use of requesting-and-freeing-of-memory cycles leads to
heap fragmentation. In many situations, control over memory management is not
needed but in the case of embedded systems this kind of resource can be determined at
design time, so buffers can be statically dimensioned for this purpose. In ICa ORB,
memory buffers can be created with a defined amount of memory so there is control
over the maximum amount of memory used for a great part of ORB operations.
5.6.4
Transport Buffering and Delays
One of the main sources of non-determinism and lack of predictability is outside the
reach of an ORB. It is the transport layer used to communicate the distributed objects.
CORBA relies in GIOP and in its implementation over TCP/IP which is called IIOP
(Internet Inter-ORB Protocol). Unfortunately, TCP/IP is not a predictable transport
and the applications developed over IIOP will suffer from the same problem.
To solve this issue, ICa ORB incorporates a pluggable framework architecture based
on the Open Communications Interface (OCI) [OMG 01c] and on the Extensible
Transport Framework (ETF) specification [OMG 02n]. Both specifications handle
similar concepts and their interfaces are based on the same architecture patterns. This
framework lets application developers build their own transport plug-ins and to attach
them to the framework. It is then possible to create deterministic plug-ins to be used
by the ORB. Several transports can be attached to the ORB simultaneously and, the
ORB will select the most appropriate one for each request according to the real-time
CORBA properties configured for the application. As will be explained throughout
this chapter, the transport framework in ICa has been enhanced to support time
attributes in its interface. The objective is to have an estimation of the time in which
actions progress in the system as needed by hard real-time systems.
118
5.6.5
Thread Dispatching
At the client-side, it is easy for the application to control the way thread priorities are
handled. In a properly configured ORB, the thread priority of the application will be
maintained by the ORB to ensure a uniform quality of service in the path to the object
providing the service.
At the server side, things begin to get more complicated (servers are CORBA’s most
complex part). A server probably manages several types of objects that share
resources and can be flooded with concurrent requests at a given point in time.
Threads to service requests must be dispatched at the server side as soon as possible
and with the due QoS attributes. For maximum efficiency in this process, ICa ORB
multithreading architecture uses the reactor pattern [Schmidt 95]. The reactor pattern
is also known as dispatcher or notifier and its purpose is to handle requests for a
service that are done concurrently by one or several clients.
From a software architecture point of view, the reactor pattern allows to employ a
simple way of thread dispatching. When software must be developed for a large range
of operating systems and devices, like the case of ICa, the use of patterns brings
programming simplicity to the software layers that use the pattern. Portability of the
software is then improved by the use of pattern solutions.
A reactor uses event handlers to service client requests. The reactor normal state is
that of waiting on a set of handlers (e.g. a socket file descriptor) by means of a
synchronous event demultiplexer (select in the case of UNIX systems). The quality of
service attributes associated to the event handler allow to efficiently dispatch an
adequate thread when activity is detected for any of the service handlers.
5.6.6
Request Dispatching
At the server side there is another problem. Servers can have many objects and each of
these objects can provide many services through their available operations.
Additionally, the objects live under the management of Portable Object Adapters
(POAs or real-time POAs in ICa). The efficient mapping of the service request that
comes from the network to the appropriate object operation is an issue that can heavily
affect performance depending on the number of active objects and on the number of
operations supported by all of them. In order to build scalable applications the
performance of this mechanism must be as high as possible.
119
The request demultiplexing or dispatching involves three basic types of object, the
ORB, the POAs and the servants.
The ORB uses the information of the object key from the Interoperable Object
Reference to find the POA that contains the requested servant. Finding the POA can
be a time-consuming task as developers may have established a POA hierarchy for the
application that must be traversed to search for the right POA.
The POA demultiplexes the request to the servant. Multiple servants can be found in a
POA. The servant demultiplexes the request to the skeleton that contains the requested
operation. The skeleton demarshals the request and executes the associated
implementation method for the operation. Notice that a servant may implement
different skeletons depending on the POA’s uniqueness policy applied.
For real-time and embedded applications it is needed that this demultiplexing
mechanism is as efficient and predictable as possible. The reader should think about
the consequences for system behaviour when the mechanisms used in each step of the
demultiplexing process do not scale well when the number of POAs, servants or
skeletons changes. The problem is that CORBA allows a very flexible dynamic
behaviour of the architecture. In the critical-systems type of applications, it is
recommended that all POAs are created at application initialisation so no adapter
activators are used. Servant activation and deactivation is another source of overhead
as no static knowledge of servants associated with a POA can be reached. For this
reason, it is desirable that no servant managers are used in hard real-time applications.
If all POAs are known at application start-up, POA Managers do not introduce
additional overhead in predictability of system execution. This is because no child
POAs are being created dynamically and work from the POA Manager is reduced to
search on a static tree of nodes to control flow of requests. This will put an upper
bound to the processing overhead for this layer of the demultiplexing process.
To perform request demultiplexing, the search algorithms below are usually employed
in most commercial ORBs. When the performance of these algorithms is considered,
the O or Φ function notation is used. The O notation is a theoretical measure of the
execution of an algorithm, usually the time or memory needed, given the problem size
n, which is usually the number of items. For instance, if it is stated that f(n) =
O(g(n)) it means that it is less than some constant multiple of g(n). As the most
relevant operation for searches is the comparison, this operation is used for the O
notation. So if an algorithm needs n comparisons to find a match it is said it is an
O(n) algorithm. If it is found in the first comparison it is a O(1) algorithm. To
120
compare the algorithms concern is put in worst-case times as knowing these times
allows to provide guaranteed performance predictions.
•
Linear search: A linear search is done on
an array or list checking one item at a time.
The average run-time is O(n/2) and the
worst-case time is O(n) as n comparisons
must be made to find a match. The linear
search is also known as sequential search.
•
Binary search: A binary search is an
algorithm to search a sorted array. It begins
with an interval covering the whole array. If
the search value is less than the item in the
middle of the interval, the interval is
narrowed to the lower half. Otherwise, it is Figure 46: Comparison of n
narrowed to the upper half. Checks are
and log n versus n
repeated until the value is found or the
interval is empty. An interval of n can be
divided in half log2n times so a binary search is a O(log n) algorithm. Figure
46 shows that for large numbers a binary search is faster than a sequential search.
The problem is that insertion and deletion operations are O(n) algorithms.
•
Hashing: Hashing is used for the object adapter demultiplexing. It provides O(1)
average-time and supports better insertions and deletions than the binary search.
The problem is that the worst-case time is O(n), being n the number of duplicates
and that may there be a high constant overhead from the hashing function.
•
Perfect hashing: Perfect hashing is based in a hash function that maps each
different key to a distinct integer. Usually, all possible keys must be known
beforehand. A hash table that uses a perfect hash function has no collisions. For
perfect hashing the number of elements must be known before hand. Perfect
hashing executes in constant time and can be compared to addressing an element
of an array through its index. O(1) is the worst-case time.
To let developers the freedom of implementing POA hierarchies and to associate
several servants to every POA a dynamic strategy is needed. ICa ORB demultiplexes
incoming client requests through a hash table at the root POA that contains
information about POAs and servants associated to a POA. The table is accessed
121
through hash functions. Once the servant has been matched a perfect hash table at the
servant is accessed to find the operation’s skeleton. In the prior steps of
demultiplexing it is not possible to use a perfect hash functions because POAs and
servants are dynamically created. For the demultiplexing of the request to an operation
a perfect hash function is used as interface operations are known at compile time.
The ICa compiler uses the GNU libg++ gperf() function to generate perfect hash
functions for interface operations. gperf() objective is to transform a keyword set W
of n elements into a perfect hash function F. F uniquely maps the elements in W onto
the range 0..k, where k >= n. If k = n then F is a minimal perfect hash function.
gperf() generates a 0..k element static lookup table and a pair of C functions.
These functions determine whether a given character string s occurs in W, using at
most one probe into the lookup table.
5.7 ICa ORB Specific Features for Hard Real-Time
Systems
In the previous section, features regarding optimization of the ORB have been
presented. Although those features are important for overall ORB behavior we should
not miss the point that hard real-time systems demand strong control of system
activities.
To provide a higher degree of control over the system service, ICa ORB interfaces
have been enhanced to deal with aspects of real-time systems that are largely obviated
by all ORB implementations. Most of these extensions to a regular real-time CORBA
ORB have to do with the management of time. In real-time systems, time represents
the hardest part to deal with because it is time the one that gives validity to what
occurs in the system. Actions of the system are valid if occur at appropriate instants of
time. In ICa ORB, there are hooks in the transport framework that allow an application
to be aware of time information regarding requests. There are also IDL extensions
designed for the handling of time stamping features and protocol statistics. The
proposed extensions for periodic tasks are dealt with in the ongoing COMPARE
project. The reminder of this chapter deals with those features in detail.
5.8 CORBA Communication Model
Most CORBA systems are based around the client-server model in which a client
makes a request and blocks until a reply from the server is received. This is the
122
synchronous two-way communication model commonly used in CORBA. The other
most commonly used communication model in CORBA is the one-way
communication in which the client does not wait for a reply. A drawback of one-way
invocations in CORBA is that, although it is not possible to specify user exceptions
for one-way invocations, the ORB is able to throw standard system exceptions which
could be raised at the server side and caught at the client side.
The invocation control that CORBA can provide should not be considered a drawback
in an ideal world. In fact, a connection oriented approach is more advantageous in any
type of application as it is possible to manage QoS and to provide isolation in an endto-end manner for the connection flow of data. The problem with the connectionoriented mechanism is that it needs error control to provide reliability and this
introduces more delay and jitter. Also, throughput between peer endpoints is more
limited as more data needs to be sent. If this is combined with the limited tolerance
that many real-time applications show to bad or lost data, the reason why real-time
application developers prefer performance before reliability is found. Invocations in
real-time applications sometimes must be pure one-way invocations with no error
control. ICa ORB is able to deal with pure one-way invocations without error control.
With this feature applied, ICa ORB can provide the same behavior at the
communications layer as that of any hard real-time system implemented in an ad hoc
way.
The Messaging specification of CORBA introduced the Asynchronous Method
Invocation (AMI) with which operations requests can be made asynchronously using
the static invocation interface. The AMI offers two ways of invoking an object
operation in a client non-blocking way, being most appealing to control purposes the
callback or reply handler model in which the ORB will invoke an application defined
reply handler upon reply from the server. This concept of callback is common in event
driven systems and has been widely implemented in different types of systems
(windowing systems like Microsoft Windows or XWindows for instance). AMI is not
part of the ICa broker but its interest for future work resides in the fact that two-way
invocations could be made asynchronously. This is a subject for future work.
5.9 CORBA Control Loop
This section introduces the basic aspects of distributed networked control for CORBA
control systems. As already explained, the lowest layer of the control hierarchy is
considered in this chapter. First, the two possibilities for embedding CORBA in this
layer are presented then, the main constraints for the timing requirements of the
control application are considered. The work for embedding ICa in the lower layer
123
control loop was founded by the European Commission’s IST programme. The project
name (IST-2001-37652) was Hard Real-Time CORBA or HRTC [HRTC 02].
5.9.1
CORBA Controllers
In order to build CORBA controllers the role of CORBA in a basic control loop must
be determined. Depending on the characteristics of the devices (control and feedback
device, sensors and actuators) that form part of the distributed control loop, a CORBA
controller can be modeled at least in two different ways:
Figure 47: Distributed CORBA Control Loop
•
Distributed CORBA Loop. This approach is the ORB everywhere approach.
In the distributed CORBA approach the sensor node, the controller node, and
the actuator node are all equipped with ORBs as shown in Figure 47. Inside
the corresponding nodes, the sensor, actuator, and controller are all modeled
as CORBA objects. Different possibilities exist with respect to which of these
objects should be active objects. Active objects are those considered as
proactive objects. They have a goal to achieve and perform actions towards it.
The first option is to let the sensor object be an active time-triggered object
that periodically takes a sensor measurement and invokes an execute operation
on the passive controller object. The execute operation typically is a one-way
message. During the execution of the execute operation the controller object
invokes the actuate operation on the actuator object passing the control signal
124
as the argument. Also this is a one-way message. Other alternatives include
making the controller the active time-triggered object who invokes in every
sample the two-way operation getMeasurement on the sensor object,
calculates the control signal, and invokes the actuate operation on the actuator
object. A drawback of this approach is the risk for sampling jitter in the
control loop.
Figure 48: Embedded CORBA Control Loop
•
Encapsulated CORBA Loop. In this approach only the controller node is
modeled as a CORBA-compliant object. The controller object implements the
communication with the actuator and sensor nodes using non-CORBA
technology, e.g. using some field bus technology (Profibus, Modbus,
DeviceNet, Genius, etc.). The controller object could be an active object or it
could be invoked from some other client representing the task execution
control of the controller. The approach is outlined in Figure 48. A similar
approach to this one can be found in the Smart Transducers Specification
[OMG 03c] that presents a model in which data can be obtained from a
network of transducers by means of a CORBA gateway.
During the Distributed Object Telecontrol Systems and Networks (DOTS) project
which was also funded by the European IST programme (IST-1999-10258), ICa ORB
was embedded into a Remote Terminal Unit device for the purpose of building
substation automation systems. The model deployed in the demonstration application
could fit either alternatives of implementing distributed control by means of CORBA.
If the application supports intelligent electronic devices (IEDs) with enough
125
computation power, the model with all nodes CORBA-enabled is used whereas the
field bus model is used in the case simple sensors and actuators must be employed.
In these CORBA controller architectures, there is always the need of dealing with
distributed systems which use a communication link to share information. Since the
feedback loop is closed over the network, the stability and control performance is
directly affected by the quality of the transport. A network that possesses too long
latencies or have too much timing variations may cause the control performance to
deteriorate or completely fail. However, if the network characteristics are well known
it is sometimes possible to take them into account when designing the control
algorithm or to compensate for them on-line. The need for a network with well-known
timing properties is evident in the design of such a networked control systems. Both
the timing properties of the network transport and of the protocol stacks must be taken
into account when analyzing the timely behavior.
Means for minimizing the network effect are needed in the CORBA distributed
system. For this, a switchable transport framework is needed in the ORB that allows
the utilization of predictable network transports where an upper bound for latency and
jitter is known. The next two sections dig deeper in the timing constraints that the
control loop must confront and into the different delays that occur in the
communications path between distributed CORBA objects.
5.9.2
Timing Constraints
The basic control loop has two main timing constraints. The first is the sampling
period, h, which should be constant, e.g., without jitter. The second constraint
involves the input-output latency, τ , from the sampling of the measurement signal to
the control signal actuation. This is also known as the control delay or the
computational delay. In a distributed control loop the input-output latency also
includes the communication delays from the sensor node to the controller node and
from the controller node to the actuator node. The constraints are illustrated in Figure
49, where it is assumed that the controller is implemented in a single task.
From a control performance point of view the following specifications are important:
•
The sampling period should be constant, e.g. the sampling jitter should be
negligible. This holds for all time-triggered control loops, which is the most
common case. An example of the opposite, e.g. an event-triggered control
loop is found in combustion engine control systems which normally are
sampled against crankshaft revolutions.
126
•
•
•
The input-output latency should be negligible or constant, e.g. without jitter.
A negligible latency can be ignored and a constant latency can be
compensated for statically in the control design.
Most control loops are more sensitive to latency than to sampling jitter.
In most cases it is better to have a small latency with jitter than a larger
latency without jitter, even if the larger latency is compensated for in the
control design.
Figure 49: Constraints of loop timing
According to the timing constraints, a CORBA ORB like ICa ORB, suitable for
distributed control purposes, must provide means to let the application be aware of the
progress of network time and to execute periodical tasks in order to provide constant
sampling of remote sensor objects and to provide actions on actuators.
5.9.3
Loop Timing
As explained before, a good control situation is obtained by equidistant sampling
intervals and a negligible or constant control delay from sampling to actuation.
However, this can seldom be achieved in practice. Within a node, tasks interfere with
each other through pre-emption, and blocking when accessing shared resources. The
execution times of the tasks themselves may be data-dependent or may vary due to
hardware features such as processor caches. On the distributed level, the
communication gives rise to delays that can be more or less deterministic depending
on the communication protocol. Some sources of communication delays are:
•
Processing delay: The time required to process the message (e.g. a packet)
within the nodes (hosts, routers, bridges) that the message passes through. At
127
•
•
•
•
•
the sending and receiving hosts this consists of the time needed to pass
through the different protocol layers (link-network-transport), whereas in
other hosts it consists of the time it takes to examine the message header in
order to decide where to direct the message. The processing delay can also
include other factors, e.g. the time needed to check for bit-level errors.
Queuing delay: The amount of time that the message spends in the output
queue (buffer) of the host or router, waiting to be transmitted.
Transmission delay: The amount of time required to transmit all of the bits in
the message into the link. For routers and bridges this is also known as the
store-and-forward delay.
Propagation delay: The time required for the bits to propagate over the link.
Transport-level acknowledgement delay: In reliable transport protocols
such as TCP, acknowledgments and resending is used to guarantee a reliable
connection in spite of bit errors and lost packets. The latter can be caused by
buffer overflow due to congestion. This source of delay can be removed if
unreliable transport protocols such as UDP can be used.
Link-layer resending delay: This is the delay caused by collision detection
and the subsequent back-off and resending in multi-access link layer
protocols, e.g. Ethernet (CSMA/CD) or CAN (CSMA/CA). This source of
delays can be removed if the network is scheduled in such a way that the
collisions are eliminated, e.g. using time-division multiplexing or through the
separate collision domains of switched Ethernet.
While the use of a predictable transport can greatly improve the behavior of the
CORBA system regarding the transport layer delays, there are some of these delays
which will always be present if the distributed system is too complex. In a heavily
distributed control system, it will not be possible to avoid the use bridges and routers
and the delays associated to these devices should be accounted for.
5.10 Event Triggered vs. Time Triggered Systems
There is a great conceptual difference between time-triggered and event-triggered
systems. In a time-triggered architecture, system activities are initiated by the
progression of a globally synchronized time base while in an event driven system,
activities are driven by other events different than the progression of time. In Section
5.8, the reply handler callback model is a pure event-triggered system as the handler is
not activated until the request from the server (the event) has been received. The
callback model or the one-way invocation communication model are suitable for realtime applications in which the communication plug-in is event-driven. Unfortunately,
this communication model alone is not sufficiently good for time-triggered systems as
128
activities are initiated / synchronized with a global time base 4 . In a system with a timetriggered communication layer, the system activities must be driven from the
communications layer and it is the transport plug-in which must be time-aware. An
event-triggered system is more flexible than a time-triggered system but the latter is
more predictable. In a time-triggered approach all activities and communication take
place at predetermined instants of time.
In practice, time triggered systems are deployed where the most stringent conditions
must be met. Avionics or automotive applications are examples or such systems. Other
real-time applications where the system is event-driven can employ transports that are
also deterministic but where the system is driven by events. In order not to reduce the
scope of applications of ICa, neither of these approaches has been left aside and two
specific transport plug-ins for real-time have been developed. A Time Triggered
Protocol (TTP) transport plug-in for time-triggered systems and a Real-Time Ethernet
transport plug-in for event-driven systems can be used to deploy real-time applications
with ICa ORB.
5.11 Distributed Synchronization
The problem of distributed system synchronisation is that of ensuring that precedence
constraints of jobs on different processors are always satisfied. This problem falls
outside the scope of this Thesis. This chapter does address neither multiprocessor /
distributed scheduling, distributed resource access control nor distributed system
synchronisation. On the contrary, it deals only with real-time optimizations and
extensions to CORBA such as those implemented for communications in hard realtime systems. Nevertheless, in a ‘classical’ hard real-time system, tasks will be
statically partitioned and assigned to processors and each node will have its own
scheduler. In the distributed system, the schedulers need a synchronisation protocol in
order to satisfy precedence constraints of tasks’ jobs. The question is what is needed
from the real-time communications in order to support distributed scheduling? As
mentioned in the next two sections, this depends on the strategy used to synchronise
the system. All the nodes in the distributed system are considered to be coordinated at
design time in the case of time-triggered applications or to have means at the transport
level to ensure that the network schedule holds (in the case of event-driven
applications).
Actually, it is not needed to be aware of the time of the global time base. Only the
difference of time with the time base and the accuracy/precision of the time base in
the nodes is needed.
4
129
5.11.1
Clock Driven / Time Triggered Systems
In a clock driven or time-triggered approach, to simplify schedulability analysis the
system is schedulable only if it is deterministic and all the tasks are periodic 5 . With
this assumption, it is possible to transform a loosely coupled system (a distributed
system in which it is costly to know global status) into a system that tries to resemble
a multiprocessor system (tightly coupled). All the nodes access to data locally which is
distributed by the real-time communication system at periodic intervals. The
communication network is as if it was a shared memory area and all the nodes
schedule their tasks independently of the other nodes. A node knows that certain data
will be available locally at pre-specified intervals in time. In this approach, no
interfaces for global system synchronisation are required.
5.11.2
RT Ethernet / Event Driven Systems
In the event driven approach, CORBA clients issue invocations as in a regular
CORBA application but requests may be buffered at the client side to guarantee that
the network schedule holds. In this approach, an RT layer is included between the IP
and Ethernet layers. The RT layer is responsible for the traffic control for worst-case
scheduling. The RT layer queries a net guard node which schedules real-time traffic in
the network. This approach does not need additional interfaces for synchronisation.
With the constraints exposed in the previous two points, nodes act as if they were
independent nodes and system schedulability depends only of the schedule of each
node. System nodes are not coupled to each other. Anyway, the schedulability of the
whole system is application bound and is not addressed in this Thesis.
5.12 The Notion of Time
In a real-time system actions or results must be accomplished before the deadlines of
the system that depend on the environment are missed. The system is called a hard
real-time system in the case catastrophic results are expected if a deadline is missed.
Flight control, brake and stability control or drive-by-wire are common examples of
such systems. Modern operating systems do not deal with the concept of time to
express tasks priorities because the complexity of the operating system and of the
underlying hardware (e.g. several levels of processor caches) makes it almost
impossible to forecast when a task will be finished. Anyhow, if the ICa ORB is to be
Provision can be made for sporadic tasks to be accomodated in the off-line
scheduling of the system but this does not fit naturally into a time-triggered system.
5
130
embedded for hard real-time applications it is needed to let the application deal with
the progression of time. This is achieved by building in ICa ORB the concept of timeaware communication protocol plug-ins.
5.12.1
Driving the System from the Communications Layer
For hard real-time applications the ORB must be aware of the progression of time.
This must hold for the distributed system. With this requisite and for a time-aware
network protocol like the Time-Triggered Protocol, time data should be obtained from
the communication layer and made available to the application or the ORB. This can
be achieved by extending current definitions for the Open Communications Interface
or the Extensible Transport Framework submission. In ICa ORB, the Open
Communications Interface has been extended to allow the retrieval of temporal
information. Nevertheless, in this document the extensions are presented for ETF as it
is the adopted transport plug-in specification by the OMG. OCI and ETF are
extremely similar and basically all the actors involved in one specification exist in the
other so translation from one specification to the other is a rather simple exercise.
A pluggable transport must be able to tell the application what time it is as well as a
measure of time precision and accuracy. In the case of periodic applications, it is also
interesting for the application to be signaled about the beginning of the next period or
cycle of execution. Proposals for the handling of periodic, aperiodic or sporadic tasks
are made later in this chapter. This kind of service is proposed as future work and will
be addressed in the COMPARE project.
It must be noticed that time-awareness for the ORB or the application might be less
precise than that of the transport protocol being then necessary to downgrade the time
for the application.
5.12.2
Interface Extensions to the Extensible Transport
Framework
In order to either drive the ICa ORB or the application activities from the
communications layer, it is needed to incorporate the notion of time into the pluggable
protocol framework of ICa. The representation of time becomes then a crucial point as
it should be simple and consume as less processing power as possible. In the traversal
approach adopted in this work, advantage of the definitions already made in the OMG
document Formal/03-01-01 Smart Transducer Specification [OMG 03c] is taken. In
this specification a time instant is represented in IDL as
131
typedef long long TimeInstant;
and its data representation is the following (excerpt from the specification):
“This type is used for timestamps. The 40 upper bits represent the number of seconds
(after 34841 years an overflow will occur) while the remaining 24 bits represent the
fractions of a second, allowing an accuracy of 60 ns. In a system with external clock
synchronization the 40 upper bits are initialized with the value 0 at 00:00:00 UTC on
January 6, 1980, which is also the reference starting point (the epoch) for GPS-time.
In this way every point in time 17420 years before and 17420 years after January 6,
1980 can be uniquely represented with an accuracy window of 60 ns. Stand-alone
systems without external clock synchronization are set to 0 during initialization.”
[OMG 03c] also defines a time duration type typedef
long long TimeDuration
“This type is used for durations that are represented in units of 2
60 ns).”
-24
as:
seconds (about
There is also a representation of an instant of time:
struct Instants {
TimeInstant instant;
TimeDuration period;
};
and its purpose is the following
“The first value (subfield instant) informs about the next instant when the most recent
of the denoted events will occur. The second value (subfield period) is the period of
the named data item.”
This representation of time can be used in the pluggable protocol framework to add a
service that allows the real-time system / ORB to learn when the next cycle of
processing / execution (read / write) must begin.
5.12.3
Where to Ask for the Time in the Pluggable
Framework?
The pluggable transport is the entity in the architecture of ICa that is aware of global
time. The pluggable / extensible transport framework provides means to forward the
time information way up to the ORB. Notice that forwarding the time to the ORB does
132
not mean that the real-time application is time-aware. The communication plug-in
interfaces are not available at the real-time CORBA application level. Then, it is also
necessary to provide an interface for the application to gain knowledge of the
progression of time.
Regarding the Extensible Protocol Framework or the Open Communications Interface,
a pluggable protocol framework handles the concepts of registry, factory, connector,
transport and acceptor (see Figure 50). Only the transport is used when the system is
working to communicate data. The reminding entities are used for configuration and
setup of connections. Once a connection has been established, it is the transport object
the one in charge of communications. This means the interface related to the transport
either in the OCI or in the ETF submission should be extended to let the ORB ask
about the progression of time. It is not possible to ask for the time without being
connected to a transport.
CLIENT SIDE
IMPLEMENTATION
SERVER SIDE
TCP/IP
Conn. Fact.
TCP/IP
Connector
TCP/IP
Transport
TCP/IP
Acceptor
TCP/IP
Acc. Fact.
Conn. Fact.
Connector
Transport
Acceptor
Acc. Fact.
ABSTRACT
n
n
1
1
Conn. Fact. Reg.
creates
Acc. Fact. Reg.
Application (ORB)
Figure 50: The ICa OCI transport plug-in architecture
Figure 51 shows the layered architecture of the real-time ICa ORB with detail of the
extensible transport framework and two transport plug-ins attached (TTP and RTEthernet transports). The time arrow shows how propagation of time values occurs in
the ORB and in the CORBA application. In this approach, the global progression of
time is only known by the transport layer (e.g. the TTP protocol) and the extensible
133
protocol framework interface provides means to allow the ORB to ask for the time.
The figure also shows that it is the message layer of the ICa ORB (the GIOP layer) the
one with complete access to the plug-in interface (e.g. to the transport object) so
without further interfaces it is not possible for the CORBA application to access
directly to the transport to learn the time.
Stubs/Skels
Time
(RT)ORB
ORB
Transport
Layer
GIOP
OCI/ETF
RT Eth.
TTP
Vendor independent
interface
Network Transport Layer
Figure 51: Source of time information in ICa
5.12.4
Particularization for the ETF Submission
To ask for the time in the ETF submission, the interface which defines the role of the
transport must be identified. In the ETF, this role is represented by the Connection
interface. The Connection interface also serves as a connector and is able to
establish connections to servers. The following IDL code is the ETF Connection
interface extended with operations to ask the global time as seen by the
communications protocol.
module ETF{
//
// declaration of time types
//
typedef long long ProtocolTime;
typedef long long ProtocolDuration;
struct ProtocolInstant{
ProtocolTime instant;
ProtocolDuration period;
//
//
//
//
//
a time instant
a duration
current time and period
time event
period of time event
134
octet precision;
// precision
};
// locality constrained
local interface Connection
{
void write(in boolean isFirst,
in boolean isLast,
in Buffer data,
in unsigned long offset,
in unsigned long length,
in TimeBase::TimeT time_out);
void read( inout Buffer data,
in unsigned long offset,
in unsigned long min_length,
in unsigned long max_length,
// transport needs to set data.length() to
// offset + number of bytes actually read
void flush();
void connect(in Profile server_profile,
void close();
boolean is_connected();
Profile get_server_profile();
boolean is_data_available();
boolean wait_next_data(in TimeBase::TimeT time_out);
readonly attribute long id;
readonly attribute boolean supports_callback;
readonly attribute boolean use_handle_time_out;
// protocol time
readonly attribute ProtocolInstant protocol_time;
// next available transmission time
readonly attribute ProtocolInstant next_transmission_time;
// time needed for the delivery
ProtocolInstant delivery_time( long size ); //in bytes
};
};
In this IDL, time is represented as in the Smart Transducers Specification. This
representation of time allows to easily synchronize a site with a granularity of up to 60
ns, e.g. with the signal from a GPS receiver (since for this time-format the epoch of
GPS-time has been chosen). The transport will provide information about the
achievable precision in the particular node.
135
For a cluster (e.g. a cluster of TTP nodes 6 ) running on its own it is not necessary to
compare the timestamps with other timestamps outside the cluster. In this case it is not
necessary to provide an absolute time base to the system. Only the differences of time
between node clocks are needed.
The ProtocolTime and ProtocolDuration have the same interpretation as in the
Smart Transducers Specification. The same holds for the precision octet which
represents the number of significant bits in the timestamp (this parameter does not
reflect the accuracy between the clocks in the system and a reference clock but just the
precision that is achieved among the clocks in the system).
Time Precision is (from [OMG 03c]):
“The Precision represents the number of significant bits in the timestamp. This
concludes in an error window of 2 39-PREC seconds. Valid values are from 0 (no
precision; the timestamp might be a random value) to 63 (an error window of about
60 nanoseconds). Note that this parameter refers to precision within an ST system, not
to the accuracy between the clocks within an ST system and the external time
reference.”
The ProtocolInstant structure is able to provide information for the current time,
a period of time and the precision of the time measure. With this data type, it is
possible to add an attribute and an operation to the Connection interface. The
protocol_time attribute represents the global time and its precision as seen from
the communication protocol. The period field is unused in this case.
For the next_transmission_time operation, the pluggable framework must obtain
timing information regarding the next protocol instant in which messages can be sent
from this node, the instant data member of the struct is the time instant in which the
next message transmission will begin, precision is the precision of this measure and
period is the period of update. Knowing the period of update, ICa clients can be
synchronised to make requests at proper times.
The third operation, delivery_time, obtains information about the needed time by
the transport to deliver a certain amount of data expressed in bytes (size parameter).
A cluster of TTP nodes is a set of computing nodes that share a physical TTP
infrastructure and are statically scheduled to provide the temporal guarantees that the
time triggered protocol provides.
6
136
The operation returns a ProtocolInstant structure in which the instant member
is the next instant in which it is possible to send a message, the period data member
contains information about the needed time to send the amount of data and
precision is the precision of the time instant.
5.12.5
Asking the Time from an ICa Application
With the extensions for the Extensible Transport Framework it is possible for the ORB
to request to the underlying communications layer the protocol time and also to
inquire when the next message slot will occur. Also it is possible to know the amount
of time needed to transmit a certain quantity of data. But the ORB is aware of neither
the purpose nor the type of messages handled by the application on top of it. For
efficiency reasons, the ORB cannot be laden with the task of actively informing the
application about timing information (a different thing is that of signalling periodic
activities). This task should be initiated by the application.
In real-time CORBA, the CORBA::Object can be configured to use a certain set of
communications protocols. This is called protocol configuration in the real-time
CORBA specification. At the client-side of the application, the policies for protocol
configuration can be applied at the object level. This means that it is possible to
establish a relationship between instances of protocols and objects in the client side of
a real-time CORBA application. It is then necessary to extend the object interface to
let the application ask for the time at the object level.
As the field of application for timing operations is vertical (only specialised
applications of a domain will make use of the interface) rather than horizontal it is not
a good idea to extend the CORBA::Object interface. Instead of this, it is better to
extend the functionality of the object in the RTCORBA module so non real-time
applications do not have the additional operations regarding handling of time.
Using pseudo-IDL the object interface is extended in real-time ICa as follows:
module RTCORBA{
local interface RTObject{
// protocol time
readonly attribute ETF::ProtocolInstant protocol_time;
readonly attribute ETF::ProtocolInstant next_transmission_time;
ETF::ProtocolInstant delivery_time( long size );// size in bytes
};
};
137
In the case of the RTObject, the time type appears as ETF::ProtocolInstant. As
the ETF also depends on Real-Time CORBA for the declaration of protocol policies it
is better if the ProtocolInstant type is defined in the RTCORBA module. In the
case of the ETF or the OCI, the declaration of the time types must be removed from
the IDL specification and an #include “RTCORBA.idl” directive is used at the
beginning of the file. The IDL follows for the RTCORBA module:
module RTCORBA{
long long ProtocolTime;
//
long long ProtocolDuration;
//
//
ProtocolTime instant; //
octet precision;
//
};
a time instant
a duration
time of event
// period of the event
precision
// protocol time
readonly attribute ProtocolInstant next_transmission_time;
ProtocolInstant delivery_time( long size ); //in bytes
};
};
The RTCORBA::RTObject interface is not derived from the CORBA::Object
interface as we are using pseudo-IDL for which inheritance is not defined.
Nevertheless, RTObject is conceptually an extension of the CORBA::Object
interface. There is a single instance of RTCORBA::RTObject per instance of
CORBA::Object. Notice that the interface extension has been declared to be local.
The extension of the interface adds functionality only for handling of time purposes
and this information shall only be valid for a given node so ICa will throw an
exception if an attempt to marshal an RT object is made.
5.12.6
Life as a RTObject
Narrowing to an RTObject is not free; there are some rules to follow when an object
reference is a reference to a RTObject.
•
•
•
RTObject instances are client-side objects.
All CORBA::Object instances may become RTObject instances.
A RTObject instance cannot be passed as a parameter of an IDL operation
nor can it be stringified. Any attempt to do so shall return in a MARSHAL
138
•
system exception with a Standard Minor Exception Code of 4 (attempt to
marshal a local object).
A RTObject narrowed to other narrower types cannot be passed as a
parameter of an IDL operation nor can it be stringified. Any attempt to do so
shall return in a MARSHAL system exception with a Standard Minor Exception
Code of 4 (attempt to marshal a local object). If a RTObject is widened to a
CORBA::Object, it is no longer a RTObject.
Making a RTObject a local object helps resolve the problem of implementing
_narrow() and _is_a(). Remember that all CORBA clients are based in the
CORBA::Object interface. Although a client object can be represented as a
RTObject, the server side will only be an Object. However, this artifact allows to
know the time for the pair object / protocol without modifying the existing CORBA
specification.
5.13 Periodic, Aperiodic and Sporadic Activities
This section has been included here as it fits into the discussion of this chapter. The
contents of this section are a proposal for the future extension (see Section 4.11.3, the
COMPARE project) of ICa ORB in the way activities of a real-time systems can be
handled. In a hard real-time system we can find three types of tasks:
•
•
•
Periodic tasks.
Sporadic tasks.
Aperiodic tasks.
In the scope of hard real-time CORBA a task is identified by a CORBA request which
in turn can start several jobs in the system to carry out a system function. It is
necessary for the ORB to identify which tasks are periodic, sporadic or aperiodic. In
this distinction, we assume that it is known at design time which tasks correspond to
each of these types. The distinction of tasks is provided as an IDL extension in the
signature of an interface operation so it can be interpreted by the IDL compiler and
information about the nature of the task can be built into the stubs and skeletons code
by the IDL compiler.
Operation declarations in OMG IDL are similar to C function declarations. The syntax
of an operation declaration is shown next.
<op_dcl> ::=
139
[ <op_attribute> ] <op_type_spec> <identifier> <parameter_dcls>
[ <raises_expr> ] [ <context_expr> ]
<op_type_spec> ::= <param_type_spec> | “void”
The optional operation attribute indicates which invocation semantics the
communication system should provide when the operation is invoked. The use of an
operation attribute is optional. The syntax in regular IDL is the following:
<op_attribute> ::= “oneway”
The one-way attribute indicates that the operation semantics are best effort, with no
delivery guarantee of the call. One-way semantics also indicates that the operation is
invoked at most once. In the proposed ICa IDL extension, the operation attribute is
extended to provide information about the nature of the interface operation:
<op_attribute> ::= [<op_model>] [“oneway”]
<op_model> is an optional operation model which is defined as:
<op_model>::=”periodic”<T>”,”<D> | “aperiodic” <D> | “sporadic”<d>”,”<D> |
Aperiodic tasks can occur in bursts and its worst execution time cannot be known
before hand. We have called sporadic tasks to those that at least are interleaved by a
delay d. In the case of periodic / aperiodic tasks, D is the hard / soft deadline of the
tasks while for sporadic tasks it is hard deadline. T is the period for periodic tasks. T,
D and d time units depend on broker implementation. It is responsibility of the ORB
to invoke periodic tasks (once the first invocation has been made) and to schedule
aperiodic and sporadic tasks 7 . For periodic tasks, it is useful to declare arguments by
reference. A new modifier for parameters has been devised for this. It has been called
ref that means that when the ORB periodically executes the periodic task the
parameter value will be actualised. For example, in C++ it can be achieved by using
references to objects as parameters.
As an example, suppose there is an interface temperature_actuator which has an
operation to periodically set the temperature value to a certain value each 30 ms. The
signature of the operation can be expressed:
This means a lot of work will have been done at design time to schedule the system.
For instance, to verify that the TDMA network schedule will provide room for
sporadic tasks.
7
140
interface temp_actuator{
periodic 30,10 oneway set_temperature(ref float temp);
};
The previous example corresponds to a periodic activity of the system. That the
activity is periodic means it has a known release time and that it has a hard deadline
(also that is executed with a known interval). The operation model attribute allows the
IDL compiler to generate stubs where this information is available to the ICa ORB. In
the case the optional model attribute is missing, it will be understood that the
operation is aperiodic. Aperiodic operations can be considered as regular CORBA
operations.
When the client invokes an operation, it is passed on to the ORB which in turn
invokes the transport plug-in to carry out the invocation. The operation model attribute
will be passed to the ORB along with the GIOP request. This way the ORB has the
possibility of scheduling the request taking into account the operation model. The
operation model also carries information regarding release time and deadline of
sporadic tasks. This is information is treated by the ORB in the same way as periodic
tasks.
In the case of periodic operations, the scheduler will be prepared to handle the
request 8 but in the case of aperiodic or sporadic operations the scheduler may perform
some acceptance test before delivering the invocation. If as a consequence of
performing the acceptance test an invocation is rejected the system will raise the
standard system exception NO_RESOURCES.
5.14 Timestamping
A RTObject can be requested to timestamp replies to requests on arrival. This allows
the application to have not only value information but timing information of the flow
of requests. The instant of arrival of a reply to a request can be known in ICa ORB. A
RTObject must be instructed to timestamp replies to requests so the overhead of
timestamping can be avoided in the ORB if the application is not handling this type of
functionality. A TimestampPolicy policy object has been incorporated in the
RTORB of ICa to support timestamping of requests.
In most hard real-time systems the periodic operations schedule is determined at
design time.
8
141
module RTCORBA{
local interface TimestampPolicy:CORBA::Policy{
};
interface RTORB{
TimestampPolicy create_timestamp_policy();
};
};
The TimestampPolicy can be used to configure a client-side real-time object via the
CORBA::Object::set_policy_overrides operation. The policy is applied only
at the client side. After setting the policy subsequent invocations by the client object
will be timestamped by the ICa ORB. An object has access only to the timestamp of
the last invocation. The functionality is supplied as an operation of ICa’s RTObject
interface.
module RTCORBA{
long long ProtocolTime;
//
long long ProtocolDuration;
//
//
ProtocolTime instant;
octet precision;
//
};
a time instant
a duration
precision
// protocol time
…
// time stamping interface
ProtocolInstant time_stamp();
};
};
The timestamp operation shall be called just after an invocation has been issued.
Attempts to invoke time_stamp without setting the policy for the object shall result
in an INV_POLICY system exception with MINOR_CODE of 1.
5.15 HRTC Protocol Statistics
Protocol properties as specified by real-time CORBA can be described in IDL for each
specific protocol. They are not defined here as it is foreseen in real-time CORBA to
provide those properties for each different protocol implementation. The protocol
properties are devised to dynamically configure a particular transport.
142
However, it has been found that in most real-time systems (mainly in event-driven
systems) a useful tool is to provide means in the ORB to gain knowledge about the
performance of the transport object. In ICa ORB, this functionality has been gathered
under the concept of protocol statistics. An interface for protocol statistics has been
added to the ORB. The HRTCProtocolStatistics interface contains transport data
which may be relevant for real-time applications. The interface has been defined to be
a local interface as it should not be marshalled outside the local process scope. The
definition of this interface follows.
local interface HRTCProtocolStatistics{
// minimum delay in microseconds
readonly attribute long min_delay;
// average delay in microseconds
attribute long avrg_delay;
// maximum delay in microseconds
attribute long max_delay;
// packet loss as probability
attribute double packet_loss;
};
To be able to take advantage of the statistics, the ORB must provide a handle for the
application to request the information. The ICa real-time ORB interface has been
enhanced with an operation to get the statistics from a particular transport. The
transport is identified by its protocol tag as several transports might be attached to an
object at runtime. The operation signature is shown below.
interface RTORB {
HRTCProtocolStatistics get_protocol_statistics(
in ULong protocolTag
);
};
5.16 C++ Source Code Example
This section shows an example of how a client-side ICa application can be configured
to use a transport protocol and how timing information can be obtained at the
application level. Most of the code is start-up code and is written only once as
bootstrapping code for the CORBA application. The excerpt of code shows how the
client-side application is able to ask for the time associated to a specific network
protocol.
// Build factory object for the new transport
//(inherited from OCI factory classes)
// only connector is needed at the client side
143
HRTCConFactory ConFactory _HRTC;
// Get a reference to the RTORB
CORBA::Object_var obj = orb -> resolve_initial_references( "RTORB" );
RTCORBA::RTORB_var rtorb = RTCORBA::RTORB::_narrow( obj );
// Get a reference to the OCI ConFactoryRegistry
CORBA::Object_var obj2 = orb -> resolve_initial_references(
"OCIConFactoryRegistry" );
OCI::ConFactoryRegistry_var ConFactReg =
OCI::ConFactoryRegistry::_narrow( obj2 );
// Add an acceptor factory object
ConFactReg ->add_factory(ConFactory _HRTC);
// Get a reference to the Naming Service
CORBA::Object_ptr nm = rtorb->
resolve_initial_references("NameService");
CosNaming::NamingContext_var nc =
CosNaming::NamingContext::_narrow( nm );
// Resolve the object by its name
CosNaming::Name name_rtObj;
name_rtObj.length(1);
name_rtObj [0].id = CORBA::string_dup( "RT_OBJECT" );
name_rtObj [0].kind = CORBA::string_dup( "" );
CORBA::Object_var object = nc ->resolve(name_rtObj );
RTCORBA::RTObject rt_object = RTCORBA::RTObject::_narrow(object);
RTObjectTest_var rt_test_obj = RTObjectTest::_narrow( rt_object );
// Set an object protocol configuration policy override
HRTCProtocolProperties props; // assigned by default
RTCORBA::ProtocolList Prot_list;
Prot_list.length(0);
Prot_list[0].protocol_type = TAG_HRTC;
Prot_list[0].transport_protocol_properties = props;
// Prot_list[0].orb_protocol_properties – defaults to GIOP
RTCORBA::ClientProtocolPolicy_var clPol = rtorb->
create_client_protocol_policy(Prot_list);
// Assign the protocol policy to an object
// after this the object uses the HRTC protocol
rt_test_obj->set_policy_overrides( clPol );
// To ask for the time a connection must be validated
PolicyList_var incosistent_policies;
rt_test_obj->validate_connection(inconsistent_policies);
144
// Get the time
RTCORBA::ProtocolInstant_var prot_time;
Prot_time = rt_test_obj->protocol_time();
5.17 The TTP Transport Plug-in
This section presents the options that were considered for the design of a TTP
transport over the OCI or the Extensible Transport Framework in order to have TTP
communications in a HRT CORBA broker (ICa ORB). TTP and its target hardware
impose some constraints on the way a transport plug-in for a hard real-time CORBA
broker can be implemented. The underlying TTP technology this transport uses has
been developed by Professor Hermann Kopetz’s group at the Vienna University of
Technology. The TTP transport plug-in was developed during the works of the IST2001-37652 HRTC project and its design and implementation as a hard real-time
protocol plug-in is part of another PhD Thesis work in our research group.
5.17.1
Main Constraint for a HRTC TTP Transport Plug-in
The TTP working mechanism consists of periodically exchanging state values
between Communications Network Interfaces (CNI). Then, the main feature to
consider for TTP regarding the implementation of a transport plug-in is that state
values are periodically copied from one endpoint’s memory to other endpoints’
memory buffers. The period of the copying mechanism is predetermined beforehand at
design time. With this consideration, there are four options available in order to
develop a transport plug-in for ICa.
1.
2.
3.
4.
GIOP messages written into CNI.
Pluggable data-interface.
Active server.
Standard CORBA + TTP/C application.
Advantages and disadvantages of either option are explained in the following sections.
5.17.2
GIOP Messages Written into CNI
TTP periodically exchanges state values between CNI memory endpoints. It is
possible to consider a GIOP message as such a state value. This way, a GIOP message
will be forwarded by CORBA clients to the transport plug-in. The transport plug-in
145
will be responsible for updating the CNI memory with the new message. During the
dedicated time slot of the node in the TDMA schedule, the GIOP message will be
copied to the other TTP nodes CNI memories. The transport plug-in at the server side
is notified of the arrival of the message and the GIOP message is forwarded to the
server endpoint ORB. At this moment, the server ORB is able to process the GIOP
message and to deliver the request to the appropriate servant. This mechanism is
shown in Figure 52. Notice that, as TTP periodically copies the GIOP message
between client and server endpoints, clients only need to make one request to trigger a
“get state value” mechanism. There is no need to make a new request if the contents of
the request have not changed. This means that in TTP requests can be executed
periodically.
CLIENT
CLIENT
ORB
ORB
GIOP
GIOP
OCI
OCI
CNI
CNI
GIOP
GIOP
Figure 52: GIOP messages copied
directly to CNI memory
ADVANTAGES
This implementation of the transport plug-in has the following advantages.
•
•
•
•
The implementation is generic. It is independent of the application as full
GIOP messages are exchanged between the endpoints.
The modification of the broker implementation only affects the development
of a new OCI or ETF transport plug-in. No changes at the ORB are required.
It is possible to pass policies, service contexts, etc. together with the request as
in a regular CORBA or Real-Time CORBA ORB. Servers can be configured
from the client-side. This makes this option more “CORBA compliant”.
Less implementation efforts compared to the other options.
146
DISADVANTAGES
The disadvantages of this approach are:
•
•
•
5.17.3
At each server a dedicated CNI memory slot is needed for each client (unicast
communication relationships). This is a limitation of the TTP cluster hardware
architecture.
Limited CNI memory (2 Kbytes) might require fragmentation of messages
(TTP packet service).
Every client needs to issue an initial request.
Pluggable Data Interface
The second option has been named after the pluggable data-interface. In this option,
the TTP communication mechanism is preserved in the sense that only state values are
exchanged between client and server CNI memory endpoints. In this case, the
implementation of the transport plug-in on the client side must preprocess the GIOP
message and write into the CNI memory only the value to be exchanged. At the server
side, the value from the CNI memory must be encoded into a GIOP message to be
passed to the ORB. This is necessary in order to maintain ORB interoperability. It is
possible to match a value with a CORBA request on the server side as in TTP each
value is written / read in a predefined memory position. Hence a value can be matched
to a request. This approach is depicted in Figure 53.
CLIENT
SERVER
OCI
GIOP
OCI
GIOP
CNI
VALUE
CNI
VALUE
VALUE = f(GIOP)
GIOP = f(VALUE)
Figure 53: Pluggable data interface
ADVANTAGES
The advantages of the pluggable data-interface are the following.
147
•
•
•
Only the transport plug-in must be modified. No changes are necessary for the
ORB.
Resources in the TTP hardware are scarce (mainly the CNI memory). This
option has the advantage that it is more memory efficient because only values
are written into the CNI memory.
No initial client request is required to trigger the exchange of data. The plugin issues server requests automatically. Clients can access state values locally
when required.
DISADVANTAGES
The pluggable data-interface also has some disadvantages. The most important ones
are described below:
•
•
•
•
5.17.4
This solution is application dependent. A new plug-in must be developed for
every TTP application. This is due to the fact that the plug-in must know that
a certain area of the CNI memory is used for a specific CORBA request so it
is able to build a GIOP request and pass it to the broker in order to invoke the
code of the servant at the server side.
Less CORBA compliant. Only state values are exchanged between client and
server endpoints. It is not possible to specify policies at the client side to be
applied at the server side or the other way round.
This option requires more computational resources to parse GIOP in the
transport plug-in. There is more processing overhead as GIOP messages are
processed twice. Currently, the GIOP layer cannot be by-passed in ICa.
Performance depends of the relationship between host speed and network
bandwidth.
Active Server
In CORBA, objects are passive entities that react only upon requests from clients. In
hard real-time applications, sampling of values (e.g. a temperature sensor) is
commonly done on a periodic basis. This means that values must be available at the
server side for clients to grab them. In a TTP application, state values can be
periodically copied from the server to the client endpoints so clients only need to
locally read the value to get the last sample. In this model, the transport plug-in at the
server side is responsible for building the request and passing it to the ORB at
predefined instants to receive back from the ORB the new value for the sampled data.
148
In this model the application at the server side is driven from the communication layer
and the server code is initiated at predetermined instants of the global time. It is
possible to think that in this approach the communication system is a client of a
passive server.
ADVANTAGES
The advantages of this approach are summarized as follows.
•
•
•
Can be combined with the “GIOP message written into CNI” or with the
“pluggable data-interface” options.
This option supports the notion of global state variables as server values are
spread by TTP across the system and are learnt locally by the client objects.
No GIOP message-processing overhead exists as the ORB and the IDL
compiler are specialized for this type of transport.
DISADVANTAGES
As happens with the other options, active servers have also some disadvantages.
•
•
•
5.17.5
Requires changes to OCI, ORB and probably to the IDL compiler.
It is the more complex option and requires larger effort to implement.
ORB interoperability is lost for this type of plug-in (no GIOP messages).
Standard CORBA + TTP/C Application
This approach uses CORBA for diagnosis and maintenance purposes. The exchange of
real-time data is performed by the TTP/C framework in an exclusively reserved part of
the respective sending slots. In a prototype, access to the real-time data is granted by
application specific code that works independently from CORBA. It is possible to
automatically generate code that performs this kind of operation without additional
programming effort from developers.
ADVANTAGES
The advantages of this approach are summarized as follows.
149
•
•
optimal temporal behavior
fully compatible with CORBA
DISADVANTAGES
The main disadvantage of this approach is that
•
5.17.6
The real-time part is not CORBA-based although it would be possible to
support this solution by a CORBA framework with code generators for the
specific TTP/C application.
Implemented Option for TTP/C
The Active Server is the best solution from a technical point of view, since it can be
built in an application-independent way. Further, the inner working of the solution
provides an acceptable performance. The great drawback of the solution is that it
requires a vast amount of changes to the ORB and the IDL compiler and that a main
premise of CORBA is lost, the ORB interoperability.
At the design and implementation stages, it was decided to have a pragmatic approach.
What is needed to prove is that a TTP transport makes the ORB more deterministic,
that it is possible to improve end-to-end predictability. In this approach, it does not
matter if a request takes a few microseconds more or less to be transmitted between
endpoints if the delay and jitter are always the same. Of course, in the real-world it
will be important depending of the type application. With these premises in mind, the
“GIOP Messages Written into CNI” approach was the first attempt to be implemented
over the TTP hardware. Due to the limited CNI memory and the quite big chunks of
information per data-value (message encapsulation), this solution was not able to
exhibit the optimal performance of the hardware regarding speed but, the results
where acceptable for a wide range of systems.
Additionally, this approach is an intermediate result for the implementation of the
“Pluggable Data Interface”. In the prototype, the application dependency is solved by
hand-written code, although in a production environment it is possible to support the
programmer with a tool that generates most of the application dependent code from an
abstract description of the application interface.
150
5.18 The RT-Ethernet Transport Plug-in
This transport uses worst case scheduling technology from Professor Karl-Erik
Årzén’s Group at the Department of Automatic Control of Lund University, Sweden.
It is one of the world’s leading groups in control theory and engineering and the plugin was developed in the works of the IST-2001-37652 HRTC project. The RTEthernet plug-in is a solution for real-time communications using a switched
scheduled Ethernet (RT Ethernet) as a pluggable transport layer within the OCI or the
Extensible Transport Framework. The main advantages and features of this approach
in comparison with standard CORBA (IIOP) and CORBA on top of TTP are the
following:
•
•
•
•
•
•
The approach is based on switched Ethernet unicast communication between a
sender (CORBA client) and a receiver (CORBA server). This is in contrast to
TTP which is based on multicast. Unicast communication has both advantages
and disadvantages compared to multicast in a hard real-time setting. A large
advantage is the better scalability and resource utilization for large
applications. Unicast communication is also the basis for standard CORBA. A
disadvantage is that fault-tolerance is not built-in.
The communication approach is invocation-oriented, in the same way as in
standard CORBA. A client that wants to send a hard-real time message to a
sender does this by making a remote invocation on an operation of a CORBA
object on the server side.
The communication is event-triggered rather than time-triggered. A GIOP
request initiated by the client corresponds to the event that triggers the
communication. However, in order to guarantee that the network schedule
holds, GIOP requests may be buffered and delayed at the client side by a realtime layer inserted in the protocol stack.
In the plug-in, only asynchronous one-way invocations are supported. Replies
to invocations are treated as separate one-way invocations in the opposite
direction. However, in principle it could be possible to also handle real-time
two-way invocations.
The RT Ethernet approach allows hard real-time communication in parallel
with ordinary “soft” CORBA/IIOP communication using the same transport.
By scheduling the access to the network (bandwidth-limited communication)
it is possible to obtain hard worst-case bounds on the communication latency.
Using the standard definition of hard real-time used within the real-time
community, e.g. guaranteeing that deadlines are met, the RT Ethernet
approach is a hard real-time communication approach. However, the jitter
latency can be considerably larger than with the TTP approach.
151
•
•
•
The approach does not require any global clock. However, if a global clock is
available the schedulability may be increased and the latency jitter decreased
if the transmissions are scheduled based on the global clock.
Due to the lower degree of temporal determinism compared to TTP, the
approach is less suited for safety-critical hard real-time applications. However,
a very large amount of hard real-time applications are not safety-critical.
Some examples are industrial robotics and automotive engine control.
The approach is based on low-cost, high-speed COTS technology. No special
hardware is required.
Using RT Ethernet for hard real-time communication involves two activities:
1. Initialization of the communication. This can be considered to be done
without any hard real-time requirements.
2. The actual communication. This is performed in real-time.
This document primarily devotes to describe the real-time communication, to a large
degree omitting the way the communication information has been initialized and set
up.
5.18.1
Switched Scheduled Ethernet
The RT Ethernet transport technique is based on switched full-duplex fast (100 MBit)
Ethernet. Every node in the network is connected directly to the switch. The
advantages of this are isolated collision domains and full duplex communication. The
only source of communication jitter is the amount of time that an Ethernet frame is
spent being buffered in the network interfaces of the nodes and of the switch. By
limiting the bandwidth of the sending nodes this buffering delay can be controlled and
it is possible to guarantee worst-case latencies. The scheduling analysis necessary for
this approach is described in [Martinsson 02].
One of the nodes in the network acts as the network guard (NetGuard). This node is
responsible for the scheduling of the real-time traffic. It also acts as a router for the
non-RT traffic on the net. In the implementation of the switched scheduled Ethernet
plug-in the scheduling functionality of the NetGuard is implemented by a CORBA
object within the NetGuard node and all exchange of scheduling information between
the nodes and the NetGuard uses standard CORBA/IIOP invocations.
152
5.18.2
Real-Time Channels
The hard RT Ethernet approach primarily considers periodic hard real-time traffic. A
periodic real-time communication between a sender node and a receiver node is
denoted a real-time channel (RTC). The real-time channel is represented by a
collection of properties describing the real-time traffic. It includes:
•
•
•
•
•
Sender node. The sender node can be identified by Ethernet address or IP
address.
Receiver node. Can be identified by Ethernet address or IP-address.
Frame size. The amount of data to be exchanged.
Frequency. The maximum frequency for the periodic communication.
Maximum latency. The maximum allowed communication latency.
If it is assumed that a global clock is available then more timing information needs to
be recorded, e.g. offsets. The Real-Time Channel is able to relay the necessary
information in order to allow the execution of a servant operation at the server side
(e.g. Object key, operation, service context, etc.).
Within CORBA the RTC maps to a connection. It is assumed that a private, preestablished connection is used for each RTC.
5.18.3
RT Ethernet Pluggable Transport
The RT Ethernet approach is structured as in Figure 54. An additional RT layer has
been added between the IP layer and the Ethernet interface. The RT layer implements
the traffic control for the worst-case scheduling.
HARD REAL-TIME CORBA REQUESTS
Hard real-time requests assume that the connection for the RTC has already been
established. Making an HRTC request generates a GIOP request that is forwarded to
the pluggable transport. The client that issued the request continues execution
immediately, e.g. asynchronous one-way invocations are used. In the pluggable
transport on the client side, the GIOP is preprocessed and the values of the method
attributes are encoded into a special RT-Ethernet frame. The header of the RT frame
indicates that this is a scheduled RT frame. This frame is then transmitted to the
server. At the server side the RT-layer receives the RT-frame, detects that it is a
scheduled frame, and passes it to the pluggable RT Ethernet transport. In the transport
153
the values of the frame are extracted and the corresponding GIOP request is created
and passed to the Real-Time ORB.
For the sake of simplicity, it is assumed that clients are “well-behaved”, e.g. that they
do not issue requests at a higher rate than what they have requested when the
connection for the RTC was established. If this is not the case, one could require that
the RT Layer should be responsible for the necessary traffic control (buffering) to
ensure this.
CORBA
Objects + Clients
RT-ORB
IIOP
TCP
IP
RT
Ethernet
Interface
Figure 54: The RT-Ethernet approach
ORDINARY CORBA REQUESTS
Ordinary CORBA requests work essentially in the same way as in ordinary CORBA.
The main difference is that the RT layer adds an additional header to the frames
indicating that this is a non-RT packet, and that the RT layer buffers and, possibly
fragments the IP packets, to ensure the schedulability of the real-time traffic. At the
server side the RT layer buffers and composes together fragmented frames and then
passes them up to the IIOP/TCP/IP layers in exactly the same way as in standard
CORBA. For the ordinary CORBA traffic two-way requests are supported.
154
The solution has a lot in common with the “Pluggable data-interface” solution
proposed for the pluggable TTP transport. The main differences are that the
communication is event-triggered rather than time-triggered and that ordinary
CORBA requests can be combined with hard real-time requests.
5.19 Platforms for CORBA Control Systems
This section is a non-exhaustive list of hardware platforms where the hard real-time
and minimum CORBA version of ICa can be run. The hardware platforms described
can be used in systems with different demands from embedded soft real-time systems
to fault-tolerant embedded hard real-time systems.
5.19.1
Embedded industrial PC Boards
This type of boards usually have a small size factor (e.g. PC104 or HDD3.5” form
factors) and provide interfaces for special equipment (e.g. GPS). Usually this type of
boards is equipped with low power consumption processors and common interfaces to
external equipment (serial, parallel and Ethernet). For obvious industrial reasons this
type of boards usually comes with a compact flash memory storage device which
serves the purpose of hard disk. Figure 55 shows two industrial PC boards with
different form factors from Lanner Electronics.
Figure 55: A PC104 and a HDD3.5'' form factor industrial
PC boards (from Lanner Electronics)
5.19.2
Networked Device Servers
Networked device servers are general purpose boards that use specialized hardware
and processors. As industrial devices, this type of server is enclosed in sturdy casing
and complies with immunity, emission and safety standards. The network device
155
servers use flash memory for the software system storage and are fully programmable.
The usefulness of a device network server relies in the variety of their interfaces. One
or more Ethernet / Fast-Ethernet ports and several serial RS232 and parallel and USB
ports are common configurations for this type of devices. RS485/422 is also
commonly found as they provide serial communication over longer distances to
devices. Figure 56 shows a network device server board from AXIS Communications.
Figure 56: AXIS Communications network device
server (front and back sides)
5.19.3
Control Units
Remote Control Units (RTU) are small computers used for automation of processes
such us control of machinery or assembly product lines. They have either modular or
integral input / output circuitry that monitors field sensors and controls output
actuators according to the programmed control strategy of the unit. They are usually
integrated into DCSs. Figure 57 shows the ELITEL-5 RTU from ELIOP where ICa
ORB runs.
5.19.4
Time-Triggered Hardware
TTP is a time-triggered network protocol based on a TDMA bus access scheme. There
are several classes of TTP protocols ranging from TTP/A for low-cost systems to
TTP/C for high speed network with high dependability requirements. TTP provides
the predictability needed for hard real-time applications while keeping fault-tolerance
capabilities.
The TTP-Development Cluster hardware shown in Figure 58 (left) is based on TTPPowernodes mounted in a rack and with one TTPMonitoring Node for real-time TTP
bus monitoring and download. Each TTP-Powernode is equipped with the TTP-C2
controller (AS8202). In addition to TTP, a broad variety of interfaces is supported:
ISO 9141 (suitable for TTP/A, LIN, and ISO-K), CAN, digital I/O, and analog inputs.
156
The TTP X-by-wire box of Figure 58 (right) is a platform for rapid development of
TTP-based distributed control systems in advanced automotive applications with highpower actuators. The box is an actuator control unit that offers hardware and software
support for direct control of a brushless DC motor.
Figure 57: A control unit from
ELIOP
Figure 58: A TTP development cluster (left) and a TTP X-bywire box (right) from TTTech
5.20 The ICa CORBA IED
Among the interesting results of and platforms employed in the work of this Thesis is
the CORBA IED (ICa PLC). The ICa PLC was developed during the IST-funded
DOTS project. Further information about this project can be found in Chapter 7. The
CORBA PLC uses the Power Substation Objects Subsystem on top of the ICa ORB
embedded in the PLC of Figure 57. The Power Substation Objects Subsystem is the
result of the mapping of the Abstract Communication Service Interface (ACSI) from
IEC-61850 [IEC 01b] to CORBA. Such a mapping from an abstract interface to a real
world mapping is denominated a Specific Communication Service Mapping (SCSM).
157
In other words, the Power Substation Objects Subsystem is a SCSM derived from the
IEC standard ACSI. As will be explained in the next chapter, the development of these
objects is an effort of domain engineering to develop a generic framework based on
ICa for the development of power substation applications. The overall structure of the
ICa PLC is shown in Figure 59.
Figure 59: UML deployment diagram of the ICa IED
A brief description of the purposes of the components shown above follows.
•
•
•
•
•
IEDApplication: it contains the main executable application packages.
ORB: it represents the ICa middleware in its version tailored for embedded
real-time systems.
OperatingSystem: it contains the VxWorks operating system packages.
AnalogPrimaryMeasure: it contains the executable application packages
devoted to the acquisition and low-level processing of analog inputs.
Configuration: it contains the configuration data of the application, in binary
format
158
•
•
•
IEDStartingCode: it contains the bootstrapping code of the operating system.
IEDLoader: it contains the packages needed to upload and download the
components located on FLASH1 and FLASH2.
ObjectsData: it contains the persistent and temporal objects used by the
executable components.
Figure 60 shows a more detailed package diagram of the ICa-based IED for power
substations. A description of the objectives of every one of the packages is presented
in the next items.
•
•
•
•
•
SAS. It is a top-level package composed of all the packages that provide the
SAS (Substation Automation System) functionality. It is composed of the subpackages in charge of the following functionality: digital input scanning,
digital input supervision, digital input fast supervision, analog input
supervision, digital output supervision and protection.
Basic Services. It is a top-level package that comprises the packages that
provide common services to the rest of the packages such as configuration
management, subscription management, event management, real-time clock,
utilities, autotest, hardware layer interface and start up procedure.
Communications. Also a top-level package that includes the packages needed
to provide communication services to the IED. The protocol stack, typically
contained (in standard IEDs over telecontrol protocols) in the communications
package, is provided by the Operating System via the ICa ORB package. On
the other hand, the drivers for Ethernet communication are implemented as
BSPs, as part of the hardware layer interface package. The communications
service sub-package provides concrete implementations of the ACSI interface
whereas the SCSMCORBA provides the mapping to the CORBA
environment.
ORB. The ORB package contains an instance of the ICa ORB for real-time
embedded systems.
HMI. Contains the packages devoted to the implementation of the local IED
human-machine interface.
159
Figure 60: UML package diagram of the ICa IED
160
Chapter 6
A Methodology for CORBA Control
Systems
161
6.1 Need of a Methodology
The final objective of ICa is to let developers build the real-time distributed systems
they envisage. But the mere existence of a framework like ICa does not ensure that a
particular working system can be built. There is a need of a methodology that helps in
the development of real-time CORBA distributed systems. Achieving the behavior
regarding QoS, dependability or safety a mission critical system might need is not
guaranteed by the simple use of ICa software. Sound engineering processes are
necessary. Therefore, best-practice guidelines must be provided by ICa to build
systems from an engineering point of view.
Further, there is also a need for software reuse to be integrated in the bottom line of
the system development process. Distributed real-time systems are highly complicated
and very difficult to build. ICa systems must be constructed from a core of software
assets that can be reused for a domain of applications. Software reuse reduces the cost,
time and risk needed for development and system deployment. At the same time, it
requires an investment for reuse as software must be specifically planned for that
objective.
Architectural frameworks like ICa only provide limited support for system design.
Consequently, system design is an issue to be solved and, as ICa is an object-based
framework, UML seems the standardized modeling language upon which an ICa
systems engineering methodology must be built. One of the most interesting features
of UML is that it is not a closed language. It makes provision for language extensions.
In the design of CORBA hard real-time systems there are two UML extensions that
are most useful; the CORBA UML profile [OMG 02m] and the UML profiles for realtime systems [OMG 04d], [OMG 05a]. These three profiles provide explicit support
for CORBA based systems and for real-time systems. Developers of ICa applications
are encouraged to make use of these facilities when planning their systems.
UML is only a modeling language and is not in itself a methodology. In defining a
methodology a software development process is needed. In some cases the modeling
language and the process are tightly coupled but in the case of UML they are not. It is
needed only that the development process is use case driven, architecture-centric,
iterative and incremental. The Integrated Control Methodology (ICm) for CORBA
Control Systems (CCS) is based upon the UML and a software development process
that respects the constraints imposed by the use of the modeling language. The
development process is based on domain engineering as the foundation for the
systematic creation of domain models and software architectures.
163
6.2 Domain Engineering for CORBA Control
Systems
In it widely recognized that system software architecture affects organizations as
much as organizations affect architectures [Bass 98] and that significant investment
must be made until a satisfactory software architecture is obtained for any particular
domain. In fact, that is the basic premise of the architecture business cycle (Figure
61), the relationship between the architecture and the organization. If this Thesis were
focused to the resolution of a specific single control problem, significant effort would
have been employed for just a single solution implementation. In engineering it is
often needed to supply some degree of generality to the solutions achieved and
precisely that is what ICm provides. It is necessary to address how generic
heterogeneous complex distributed control systems are developed following software
architecture practices in the domain of distributed control systems.
Architect’s Influences
Architect
•Requirements and qualities
))
•(end user and developing organization
Architecture
•Technical environment
•Architect’s experience
System
Figure 61: System architecture business cycle
To be able to generically develop for a range of systems, they must share a common
reference model. In most DCSs there is an underlying common model that allows us
to build systems by specifying the set of common software components to be used and
the data flows between them. In this context, the reference model becomes a reference
architecture. If a core set of software assets is developed to implement systems, the
reference architecture becomes the software architecture for a type of systems. This
means that DCSs product lines can be built as product families. Construction is made
164
by reusing the core software artefacts that represent the assets of a software product
family 9 .
Nowadays, there are different industrial areas where DCSs share reference models.
This is something that occurs frequently in mature domains. In many cases, there is
also agreement in the reference architecture for a domain. Unfortunately, in the DCS
industry, there is not such an agreement regarding the implementation of the
architecture and thus, thousands of different pieces of software are used for every
system implementation. This has a direct relationship with the architecture business
cycle. In some cases, there are system requirements that force the use of certain
elements for technical reasons (e.g. a real-time operating system) while in others, the
factors that affect the mapping of architectures to systems can simply be related to
economic stakes (e.g. raising of commercial barriers for competitors). This affects
system architecture and, as shown in the architecture business cycle, the architecture
will affect the buying organisation which will be married to a developer / vendor and
will suffer for increased maintenance costs among other drawbacks. The only way to
avoid this is to think in architectures that are implemented in a standardised way.
One of the benefits of standardisation is that it forces the people involved in the
standard specification process to think about multiple systems, they have to meta-think
in what systems need and to provide common interfaces for the domain of application.
This is the basic idea of domain engineering. In this and the previous chapter, it has
not been forgotten that there is work in progress in the OMG and that there are
hundreds of people that may already be working in the specification of solutions to
problems that exist in the distributed control systems domain. Thus, in order to build
ICm an effort has been made to find which CORBA specifications and services can be
employed in the control target domain. The final idea is to organize a collection of
reusable software assets to be employed in CORBA Control Systems. A primary
effort has been a traversal research across domain specifications. This is a typical
problem of large standardisation or specification bodies that group large amounts of
people. It is difficult to know what other groups are doing and in occasions it results
that the same problem is solved independently by several groups.
Although rework can be avoided in some cases just by learning what others do, it was
found out that there was not an organised effort into the OMG to deal with the issues
of distributed control systems. In order to develop core assets for control systems, the
A product line is a collection of products for a market niche. A product family is a
collection of products that share assets (designs, components, etc.). The ideal situation
is the construction of product lines as product families as this maximizes the
effectiveness of the engineering process (time, cost and quality).
9
165
ICa infrastructure was developed to build control systems upon it. In this case, the
infrastructure, ICa ORB, is a real-time CORBA compliant ORB enhanced for DCSs.
Although CORBA was not initially foreseen for industrial systems, it has
specifications for real-time and fault-tolerance. Unfortunately, these specifications,
specially the real-time specification, miss to treat some of the most important points
for control systems. To overcome these difficulties, a completely new CORBA ORB
was developed to incorporate functionality for control purposes. Specifically, as
explained in the previous chapter, many areas of the control domain have been
addressed: sources of overhead or non-determinism in the ORB, communication
models, CORBA control loops, event-triggered systems versus time-triggered
systems, the notion of time in control systems, etc. (see all details in Chapter 5).
With these items considered, it is possible to have an infrastructure upon which core
assets for control systems can be built. It is not possible to have assets with real-time
or other control properties if the underlying system, the middleware, does not support
them.
In an attempt to formalize the process of designing for reuse and making it more
repeatable, verifiable and methodical, the focus has been put on domain engineering
[Harsu 02]. Domain engineering, when compared to application or single-system
engineering, has, at minimum, these distinct aspects:
10
•
Multiple System Scope. Control system engineering involves designing the
best solution for a single intended application (e.g. a control system for a
specific system to be controlled). CORBA Control Systems domain
engineering involves designing assets that will optimise –or at least enhance–
the overall engineering process for some set or class of distributed control
applications. Unlike developing a single control system, where there is at least
in principle some optimum (or at least adequate) single solution to be
designed, in domain engineering there is an attempt to model the space of
solution alternatives themselves.
•
Smaller than System Scope. Control systems engineering, presented with a
single application context, implicitly has the job of designing the “whole
solution” 10 . Domain engineering, while spanning multiple applications, has
the “luxury” of picking what scope of functionality to address across those
systems. For example, a domain engineering effort addressing the family of
custom applications created within a telecommunications company could
Obviously for the context of application.
166
focus specifically on the software implementing a specified class of features
(e.g. call forwarding, call waiting) within those overall systems.
This means the CORBA Control Systems domain engineer has several scoping
decisions that are not part of the ordinary control systems development process:
scoping the “range of coverage” for the components to be developed, both in terms of
the type and number of control systems within which the components will be
applicable (the “market” or reference model) and the specific features of those systems
to be addressed by the components (the “coverage” or reference architecture). If these
and other related methodological issues are not dealt with intentionally in the process
followed, they can compromise what would otherwise be workable software
engineering results. These basic concepts therefore turn out to have pervasive impact
on all the phases, activities and work-products of the domain engineering life cycle.
As with any work done with some perspectives of future, several overlapping and
nested domains must be confronted. CORBA object-based reusability empowers
developers with what we call progressive domain focalisation [Sanz 99a]. This means
that development can be focused in a domain without sacrificing extensibility to
broader domains and without hampering further concretion in narrower domains.
6.3 Methodology Overview
The ICm methodology for CORBA control systems is based on domain analysis,
design and implementation. Working on domains instead of applications or systems
yields several advantages:
•
•
•
CORBA control systems analysis collects relevant information from existing
systems of a control domain. The analysis identifies commonalities and
differences and provides understanding of the relationships among the
components of the domain. Most important, analysis bounds the domain.
The domain model resulting from analysis is used to carry out CORBA
control systems domain design. The main product of domain design is a
generic architecture for CORBA control systems in a certain domain. The
generic architecture arises from the domain knowledge gained in the analysis
phase and from the study of existing classifications of common architectural
styles and system patterns.
CORBA control systems domain implementation consists in the elaboration of
patterns and repositories of reusable components for the control domain.
167
Components of the domain have been identified from the analysis and design
phases and from its main product, the architecture for the domain.
In accordance with domain focalization, the proposed methodology allows to develop
different types of solutions for CORBA control systems. They have been classified
according to the different types of entities that can be constructed while developing
CORBA Control systems. The solutions are classified as follows.
•
•
•
•
ICa Objects. They are the basic components of a CORBA Control System
solution. Objects can be thought of as software objects in the sense of Object
Technology and can be distributed (CORBA objects).
ICa Patterns: They are basic object organizations that provide some abstract
functionality. They are instantiated in real-time systems (typically by means
of ICa Assets).
ICa Assets. They are software artifacts made of composed ICa Objects. While
objects provide simple functionality, assets are able to offer complex services
to systems or subsystems. Tipically are pattern reifications.
Systems and Subsystems. Systems are composed of assets. A system is a
CORBA Control solution that is made of from the integration of assets.
In ICm composition is achieved by means of sticking together the listed solutions by
use of the ICa platform middleware. The processes involved in the creation of
CORBA Control Systems are depicted in Figure 62.
Product
Line
Analysis
System
Systems
Domain
engineering
Application
engineering
Commodity
Assets
Objects
Principal
ICa
Figure 62: Integrated Control methodology process
168
6.4 CORBA Control Systems Domain Analysis
Domain analysis in ICa includes domain scoping and identification, the systematic
analysis of commonality and variability for a family of systems across the distributed
control domain and the construction of the feature or requirement set for the family of
systems. The objective of the analysis is to identify the elements that are common to
all the members of the CORBA Control Systems family and to identify those elements
that are variable across members of the family. The first step into domain analysis
consists of the identification and scoping of the domain of application.
6.4.1
CORBA Control Systems Scope
As a first step of the methodology analysis phase, the engineering domain must be
identified and bounded. Usually, domain focalization must be used to dynamically
change the scope among the different elements that form the system (subsystems,
assets, objects). Shifting the scope gives the architect or engineer the chance of
reducing the size of the domain and to concentrate on the task and technology that
must be employed for a subsystem. Further, domain focalization helps to find
solutions that are valid across domains.
During this chapter, ICa and other systems will be used as examples of the use of the
methodology to build a framework for the family of CORBA control systems. Thus,
the ICa methodology will be used to explain the domain engineering behind ICa. ICa
technology is a system class solution in the list of solutions that the methodology helps
to achieve. However, this is a solution that has been achieved as a result of domain
engineering and application engineering for a family of systems as ICa provides a
framework for the development of solutions in the CORBA Control Systems domain.
ICa is used as the core example of scoping the domain for a family of systems. The
scope of ICa is that of distributed control systems. This is an enormous domain and is,
perhaps too, simply defined. It is because one of the objectives of this work has been
to demonstrate that a single technology can be employed to address many problems of
distributed control. Thus, the ICa scope comprehends from the smallest embedded
distributed control systems to large enterprise / industry wide control systems.
ICa technology (broker, objects, assets, etc.) can be used for a wide range of
distributed control systems because it has been designed to support a high number of
points of variability that are used to implement specific control systems. Domain
169
engineering (see Section 5.2) has been used to identify points of variability in the
distributed control systems domain.
6.4.2
ICa Domain Analysis
The most difficult part when performing domain analysis is to identify the points of
commonality and variability for a family of systems and to cluster them into a feature
set. ICa provides a framework in which points of variability allow to derive specific /
generic solutions to control problems by means of application / domain engineering
based on the common vision provided by the ICa ORB. Commonalities are less
difficult to determine as common components of systems can be in most cases easily
identified by the examination of multiple systems used as exemplars.
As shown in Figure 62 different types of solutions can be developed by the use of the
ICa framework. This is because ICa has been thought as a generic framework for the
whole range of distributed control systems. Thus, with ICa and its methodology it is
possible to
•
•
Perform domain engineering to develop a framework on top of ICa to solve
the problems of a specific control domain. For example, domain engineering
can be applied with ICa to develop a framework for time triggered control
systems. This means knowledge in the time triggered systems domain is used
to develop an infrastructure for a product family.
Perform application engineering on top of ICa to develop specific control
solutions. This can be achieved by hard coding the points of variability in ICa
to solve a specific control problem. This is exactly the same as customizing
the ICa framework to develop a distributed control application.
Domain definition and scoping is one of the hardest parts of domain analysis. Domain
analysis uses existing examples and the knowledge from domain experts and other
sources of information to provide a domain model from which to instantiate
application systems. However, in the case of distributed control systems this model is
very complex as the domain is very wide. At the same time, the domain model does
not describe how good control systems are built. This problem has been solved in this
work by developing both the ICa ORB framework and the ICm methodology. The
methodology helps capture domain knowledge and to plan software with / for reuse
while the ICa framework is the means to instantiate the objects, assets and systems of
the domain and gives to developers generic points of variation for distributed control
systems design.
170
For this approach to work, the ICa broker itself has been developed by using domain
engineering so it supports the features needed for the distributed control systems
domain. Notice that we have specifically used the word “features” as opposed to use
cases. ICa ORB provides features to objects, assets and systems built on top of it.
These elements can be part of other domain engineering efforts (domain focalization
has been used to narrow the focus of ICa) or can be developed to implement single
systems by direct application engineering on ICa. The approach is presented in Figure
63.
Control System A
Generic Control
Domain Engineering
Use cases
Control Framework A
Use cases
ICa
features
Use cases
Specific Control
Domain Engineering
Control System B
ApplicationEngineering
Figure 63: Domain engineering and analysis with ICa
In the figure, ICa features drive the methodology towards development of control
frameworks and systems or applications. At the core, ICa features have been achieved
by domain engineering of the distributed control domain. This domain engineering is
labeled as generic as features are provided for a wide span of distributed control
systems. From the core of assets that ICa provides, traditional domain analysis can be
employed to focalize on other control domains, e.g. telecontrol of power substations.
Likewise, by performing application engineering it is possible to directly implement
systems with ICa. Chapter 7 provides examples of real systems that have used
specific control domain engineering and application engineering on top of ICa.
171
The advantage of this method of analyzing a domain is that ICa has already performed
a great deal of the domain analysis. When designing a distributed control system, a
great deal of decisions must be made. These decisions pertain to different domains: the
control systems domain (and its sub-domains, e.g. the automation systems domain or
the embedded systems domain, etc.), the distributed systems domain (in the case of
ICa, this is fundamentally the CORBA domain), the real-time systems domain and
finally the domain whose problems the domain engineer needs to deal with. ICa and
its methodology allow the engineer to focalize only in the domain he needs to address
(for instance the control of power substations domain) and forget about the remainder
domains as they are already contemplated by the ICa framework. The only work to be
done regarding the collateral domains is to select the needed features from ICa ORB.
To allow developers concentrate on a specific control domain, the ICa ORB domain
analysis has decomposed the problem of building distributed control systems (see
Chapter 5) trying to provide maximum satisfaction regarding the following
requirements:
•
•
•
Control over processing, communication and storage resources to improve
real-time features.
Variable framework size to be fit into disparate target systems.
Integration in heterogeneous environments.
In the process of domain analysis and as ICa is an object oriented framework, it is
recommended that UML use case notation be used in determining the commonalities
and variability of the domain. Use cases, when expressed graphically, are not good at
modeling variability because there is not a simple way to specify optional features
(e.g. a control system is / is not supervised) or variant features (alternative ways to
configure a mandatory or an optional feature). Use case modeling is recommended but
always including its narrative form. The important thing is to correctly convey the
information of the analysis. There are no silver bullets regarding notations for
variability and frequently several techniques are needed depending on the final user of
the information. [Trigaux 03] and [Bittner 03] provide a discussion of the basic
techniques and their advantages and disadvantages.
6.4.3
Steps of CORBA Control Systems Domain Analysis
with ICa
As a summary of the steps involved in the domain analysis for CORBA Control
Systems with ICm the following list is provided. A graphical representation of the
172
process is shown in Figure 64. Examples are given concerning the particular domain
of Substation Automation Systems.
1. Identify the specific control domain. The distributed control systems domain
is very wide. ICa has been designed to provide support for the whole range of
applications in the domain. However, when developing for a family of control
systems the domain shall be narrowed (domain focalization). An example of
this process is found in the DOTS project (see Section 7.2.3) which develops
a framework for Substation Automation Systems (SAS) on top of ICa. The
designer only needs to focus on design for the SAS family of control systems
while ICa provides a feature set common to SAS and other DCSs that can be
reused with a limited effort.
2. Select analysis examples and identify the feature set for the control
domain. To this respect, the commonalities and variability in the control
domain (e.g. SAS) must be factored out. The common way of doing this is by
examining a set of systems that adequately represent the domain. In the case
of the DOTS project, the necessary knowledge was obtained from the experts’
knowledge that was put into the IEC-61850 standard for protection and
control of electricity transmission and distribution systems and from domainspecific industrial project partners.
3. Cluster the features of the domain. Features must be clustered into layers,
subsystems, assets or objects. This is often a hard part of analysis as heavy
domain knowledge is needed. At this point domain entities, their behavior,
interactions and functionality must be known. This process is carried out at a
block level. Precise descriptions (e.g. interfaces) will be carried out in the
domain design phase. It is possible that some of the identified entities are only
analysis entities that will be transformed in design and implementation entities
in later phases of the development process.
4. Identify the features that are supported by ICa ORB. Although with a
great degree of flexibility, ICa guides the overall design of the reference
architecture for a family of systems. This means that, at the end, the
components of the family of systems are mapped to CORBA objects running
on top of the ICa ORB. It is important to determine what features are directly
supported by ICa. For instance, if a component needs to be multithreaded it
will be necessary to use a threadpool and to associate it to the portable object
adapter where the component runs. This implies that some design decisions
are mapped directly to the ICa framework and, that the developer need not
worry about them. More difficult-to-meet features are also supported by ICa.
In the case of the DOTS project, requirements from IEC-61850 such as “The
functions used today are well known but the standard shall be open for future
173
requirements” are met by ICa intrinsically. As a CORBA ORB extensibility,
adaptability, scalability and other features are core concepts of ICa ORB.
Focalize
domain
Identify
feature set
Cluster
features
Identify
Ica features
Steps of ICa Domain Analysis
Figure 64: Phases of ICa domain analysis
6.4.4
Example of ICa Domain Analysis for SAS
This section presents a summary of how ICa domain analysis has been applied to the
domain of Substation Automation Systems (SAS) in the scope of the DOTS project.
The analysis followed the steps designed in the methodology for this purpose.
Domain identification. As it is easily understood several domains overlap. Pure
domains are very rare to find as usually there is common functionality that is useful
across domains 11 . This is what ICa takes advantage from. In the case of building a
framework for SAS, the domains are the following: distributed systems domain, realtime systems domain, reliable systems domain, embedded systems domain and the one
with real interest for end users, the SAS domain. ICa relieves the pain from developers
for all domains except from SAS (with limited support for reliable systems). Then, it
is possible to focalize on the problems of substation automation and control. Figure
65 shows an overview of the complexities of the SAS domain. The figure shows a
collection of components (from IEDs to remote supervisory control) that can take
advantage of ICa to provide integration in a standardized way.
11
This is in fact the reason why a transversal approach like ICa can have any success.
174
Operation terminal /
Configuration terminal
Remote
RemoteSCADA
SCADA
Front end
High speed
links ,network
Low speed
links
IED
RTU
Maintenance
terminal
Physical
devices
Controlled Plant
Figure 65: General view of the SAS application domain.
Feature set identification for the SAS domain. For this purpose it is necessary to
take into account exemplar systems, domain experts knowledge, common systems
requirements and all the information available to derive a feature set for the
applications in the domain. In the case of SAS and for the DOTS project, reliance was
put in the feature set identified by the IEC-61850 experts committee. Features in this
IEC standard are grouped under the concept of function. A function is a task carried
out by the SAS which exchanges information with other functions to carry out its
work. With this information it was possible to develop a set of features related to the
SAS functionality: control functions, metering functions, disturbance recording,
protection functions, switchgear functions, etc. Figure 66 shows how SAS
functionality is structured at the different levels of the substation.
SAS feature clustering. Fortunately, in the case of the DOTS project, SAS features
were already classified (at a high block level as needed in domain analysis) by the
IEC-61850 standard. This clustering together with a grouping of feature classes is also
shown in Figure 66. The figure clusters the features according with the different
levels (layers) identified in a substation (e.g. the process, bay and station levels).
Notice that in the case of distributed control systems feature clustering is usually
influenced by the physical devices of the system and by the layered architecture of
distributed control systems. A finer clustering of features is achieved by examining the
175
functionality of each of the devices. A higher level of feature clustering can be
achieved by examining the way devices collaborate to perform systems functions. This
last case is what is considered for the domain of SAS.
Features identification. For the given analysis, the features to be provided must be
sorted out. These are the features that may be found in most distributed control
systems. In the case of SAS, these features were related with the capability of ICa to
be integrated with other (legacy systems or not) systems, its capabilities to be
embedded in devices, and with their abilities to control concurrency and
communication transport aspects of the system.
The completion of the four steps explained before is what makes up domain analysis
for SAS. After this, it is possible to employ the information of the analysis to devise
an architecture for the domain. That part of the methodology corresponds to the
CORBA Control Systems Design of the ICa methodology.
Figure 66: General structure of a substation automation system
as shown in the IEC 61850 Standard [IEC 01c]. Level 0 =
process, level 1 = bay, level 2 = station.
176
6.5 CORBA Control Systems Domain Design
The objective of domain design is to take the results of the ICa domain analysis and to
elaborate them to produce a generic design and implementation for a family of related
systems. To produce such a design it is needed that the abstraction level employed
during the analysis stage is kept also at this stage (in the sense that design is not being
made for a single system). Design must result in an ICa-based software architecture
that is reusable across systems.
As happens with all software architectures, the resulting domain model of the design
process must supply two things:
•
•
A detailed partitioning strategy that defines the components of the architecture
and the allocation of the domain feature set to them.
A coordination and control model that defines the behavioral model of the
software elements that compose the architecture.
The former steps must be performed taking into account that ICa is based on an ORB.
6.5.1
ICa Domain Design and Implementation
As said, while doing analysis, it is important to remember that ICa ORB is a CORBA
ORB so architectures resulting from domain design will be influenced by the reference
architecture of the OMA. However, the trick here is to remember that ICa is a
framework for frameworks. This means that it is possible to build other generic
architectures for distributed control systems on top of ICa (see Figure 63).
In the ICa domain analysis, a partitioning strategy was already carried out. It was
called feature clustering. That partitioning strategy was done in a coarse-grained style.
Usually, in the type of systems which are developed with ICa, that partitioning
corresponds to physical entities of the system. For instance, when analysis was made
in the scope of the DOTS project, feature clustering was focused around the real-world
entities that appeared in the system. Remote terminal units, operator terminals, switch
gear, etc. were some of those entities. However, when going deep to design, feature
clustering is too high-level. Then a more detailed partitioning strategy that deals with
processes, interfaces, objects, connectors, control flow, etc. is needed.
The second aspect of ICa domain design is the coordination model. It is not possible to
design a good coordination model if the domain is not properly partitioned.
Development with ICa means developing with distributed objects. If the entities of the
177
domain (objects) either are not well chosen or are too poor there is significant chance
that the coordination model will not have the sufficient flexibility to fit a family of
systems. On the other hand, if the domain object model is too rich, there is also a
chance that the coordination model will be too complex. In this case, if the system is
heavily distributed, this will result in a loose of performance due to excessive use of
communications. Additionally, devices with scarce resources will be overloaded as
they will have to instantiate more objects than needed. A good rule of thumb is to
remember that sending a message to an (distributed) object is very expensive in terms
of communications and computing at both sides.
The exact richness that must be conferred to the domain model is a matter of
experience. Too poor the domain model and the benefits of object orientation could
not be reached. Too complex and the responsiveness of particular implementations
will be a problem.
The last point in ICa domain design is that of designing for reuse. Pure domains do not
or rarely exist and most systems exhibit features that can be found in several domains
at the same time. This is why ICa is useful; it takes advantage of features that are
useful in different domains (e.g. the sub-domains of distributed control systems).
Designing for reuse means planning in advance for reuse. The best recommendation
that can be done for reuse is to keep an eye for elements of the architecture which are
dependent of each other. Dependencies preclude reuse as it is a sign that the elements
are tightly coupled.
Next, the steps of ICa domain design are presented.
6.5.2
Steps of CORBA Control Systems Domain Design and
Implementation with ICa
The main results of ICa domain analysis were the identification of the domain, feature
identification and the clustering of features (usually around devices) and the search of
features that could be implemented by ICa. Here, the steps of converting those results
into a generic architecture and implementation for a family of systems are introduced.
1. Subsystems identification. When developing with ICa, systems are made up
of smaller sub-systems with defined missions. The first step consists of
subsystem identification. This step is achieved by assigning clusters of
features to subsystems and defining the different structures that make up the
system. These structures can be conceptual, process, physical, control-flow
structures or any other considered important from the software architecture
178
point of view to identify each subsystem. Whether the system is made up of
subsystems or just ICa assets is matter of system size.
2. Asset identification. Assets are the primary means of reuse and subsystem
construction. While at the subsystem level it is difficult to define interfaces
that can be employed many times over, at the assets level there is more
independence of the external constraints of the domain. So in the long run, it
is better to design for reuse at this level (patterns play an important role here).
An asset is a set of objects that collaborate to perform a system or subsystem
function. Collaborating assets build up subsystems.
3. Object identification. Objects are the basic elements of ICa development.
Objects collaborate to form assets. Thus, objects have only basic behavior. In
creating objects, common object techniques can be used. Object attributes and
persistency must be clarified as well as their operations and the degree in
which the operations may vary so assets exhibit flexibility and can be used in
different domains. Notice, that when moving from analysis to design, analysis
objects can be decomposed into several collaborating design objects.
4. Interface types identification. So far, the system is defined by sheer
definition of its constituent parts from different points of view. However, the
interfaces of subsystems, assets and objects may be realized in two different
forms: remote interfaces and local interfaces. Remote interfaces correspond to
ICa objects while local interfaces cannot be directly accessed by ICa. The
point here is that distribution is always expensive and should not be employed
unless necessary. Additionally, making all system elements interfaces
remotely accessible increases system overhead in performance and size so
remote interfaces as small as possible must be defined. In general, objects do
not have remote interfaces unless the system is small, the object is
independent of other objects and the object performs functions which must be
accessed from other distributed objects or assets. Assets tipically have one
remote interface for the whole set of objects that form the asset. Similarly,
sub-systems have one interface that allows access to subsystem functionality.
Notice that lower levels may not have a remote interface if they are part of
upper level elements (e.g. an asset may not have a remote interface if it is part
of a subsystem and all the assets that make the subsystem run in the same
process). A final note is that of programming style. When dealing with local
interfaces, fine-grained interfaces can be developed (e.g. including get or set
operations / methods for object attributes) as the overhead of sending a
message between objects inside a process can be dismissed. However, remote
interfaces must exhibit a coarse-gained programming model. Each operation
of an interface must contain all the arguments needed to perform complex
functionality. This will minimize network usage although the programming
model is more involved.
179
5. Create behavior 12 . The pre-final step consists in designing the allocation of
the features identified in domain analysis to subsystems, assets and objects.
For this, the distinction between local and remote is vital, the fewer remote
components in the system the better. Behavior is also related to
implementation as determining the capabilities of the domain framework in a
specific application is a matter, for instance, of the device resources where the
application runs. Thus, in many cases a reasonable option is to devise the
framework in such a way that ICa behavior can be parameterized for later
application engineering.
6. CORBA Control Objects framework implementation. At this moment, a
distributed object framework for a family of distributed control systems
pertaining to a domain has been devised. It is time now, to proceed with
implementation. Notice that analysis and design of the domain objects has
already been done and that this step is devoted to the implementation and tests
of framework objects of the domain (e.g. those objects of a domain layer) and
not those corresponding to a specific application. The problem is that the
framework implementation must be used in a variety of devices with different
amount of resources. In this case, some domain engineering for control
devices inside of the main process of domain engineering for control systems
is needed. A device abstraction layer is needed so a common interface to all
the devices of the domain is designed. Such an interface isolates the domain
objects from specific hardware and operating system details.
Notice that in the previous steps all the design is related to object oriented design.
Figure 67 shows a graphical representation of the steps associated to design and
framework implementation. Objects are not standalone parts of a system. Objects
collaborate and thus the coordination model of the system is devised along the
partition model of the system. In other words, a system cannot be split into objects
without thinking how those objects (be them the subsystem, assets or ICa objects)
coordinate.
6.5.3
Example of ICa Domain Design for SAS
In continuation of the example relative to domain engineering for Substation
Automation Systems, the steps representing the ICa domain design for a framework
for SAS is presented next.
In practice, steps 4 and 5 can be overlapped in time as system behavior can be used
to obtain interface types.
12
180
Identify
assets
Identify
subsystems
Identify
objects
Identify
interfaces
DocumentList
FileMgr
UI
Repository
Document
add( )
delete( )
fetchDoc( )
sortByName( )
DocumentList
name : int
docid : int
numField : int
get( )
open( )
close( )
read( )
sortFileList( )
create( )
fillDocument( )
MFC
FileList
fList
FileManager
add( )
delete( )
read() fill the
code..
1
DocumentApp
Document
rep
Repository
RogueWave
GraphicFile
(from Persistence)
File
Persistence
File
read( )
GrpFile
name : char * = 0
FileList
readDoc( )
readFile( )
read( )
open( )
create( )
fillFile( )
global
Subsystems
Objects
Interfaces
Implement
framework
Create ICa
behavior
Framework
implementation
ICa behavior
Assets
Steps of ICa Design and Implementation
Figure 67: Phases of ICa domain design and implementation
Subsystems identification. When the example of domain analysis for SAS was
introduced, a coarse-grained partition of the SAS system was made. Basically, the
feature set for the domain was identified and then, the features were clustered into
layers of similar functionality. For SAS three layers or levels in SAS terminology
were identified. These were the process, bay and station levels. The subsystem
identification digs deeper into these levels to identify the subsystems that compose
each of these levels. As an example of subsystem identification, focus will be put on
the bay level or Level 1. Level 1 corresponds to the bay units, the PLCs or IEDs
placed on top of the process level. Here, for the subsystem identification, the point of
view has shifted slightly from the mere functional features of the system, as
considered during the analysis, towards a composed view that includes also
implementation aspects of the layer to be built. Figure 68 shows how implementation
(at the design level) and functional features of the system are related.
181
Figure 68: SAS Level 1 subsystem identification
For the subsystem identification of the layer it was important to decouple SAS
functionality from the communications technology to be used. This is the same as
separating the ICa ORB technology from the SAS functions of the PLC. The upper
figure shows the SAS, Communications and ORB packages (or subsystems if an UML
stereotype is used) designed with this purpose. The HMI subsystem controls the
specific human machine interface of a particular PLC or IED. The separation between
the SAS, Communications and ORB subsystem is important as a framework for SAS
based upon ICa technology is being built and the advantages of the real-time CORBA
technology must be preserved. The separation of subsystems helps to modify SAS
functions without bothering about the Communications and ORB subsystems (as long
as the change does not affect distributed objects). Likewise, the Communications and
ORB packages could be changed without the SAS functionality being affected. For
182
instance, the framework does not depend of any given ORB vendor. Also important is
the separation between the SAS subsystem and the Basic Services subsystem. The
latter subsystem is close to hardware and to the operating system and isolates the SAS
functionality so it can be reused in different devices with different capabilities. In the
case of the ORB subsystem, the ORB package is the real-time version of ICa ORB.
The choice of the real-time embedded version of the ORB is imperative because a bay
level unit in a SAS application is involved in critical distributed operations of the
substation (e.g. interlocking) in an environment with limited resources.
Assets identification. Assets identification is also shown in Figure 68. Inside each
subsystem, the assets have been denoted as smaller UML packages embedded into
subsystems. When analysis was considered, it was said that assets were the basis for
reuse. In this case, assets are related to SAS functions and thus, a specific IED or PLC
can be tailored to support or to lack some SAS functions. This way a product line for
SAS with products differentiated by features can be created. Notice that the
Communications package depends of the SAS package. Basically, the contents of the
Communications package change by recompilation of the ICa IDL files needed for a
device of the product line. The stub code is automatically generated by the ICa IDL
compiler.
Object identification. Object identification is a matter of focusing into the assets and
finding the entities that collaborate for each SAS function. This is much an effort of
object oriented design. Design classes from OOD must be found in this step. Notice
that domain design is being dealt with in this section but, at the same time, object
oriented analysis and design must be done here. Moving from object analysis classes
to design classes creates new entities in the system. For instance, how dependent
subsystems communicate must be elucidated. The use of the façade pattern for
subsystem interfaces is a good means for solving this type of dependencies.
Interface types identification. Once the design classes (or runtime objects) of the
assets, subsystems, etc. have been identified, it is time to work out their interfaces.
This includes understanding the dynamic behavior of the objects in the domain and
deducing from it the exchange of object messages. In the case UML is used, the
sequence of messages that conform the interface of an object can be deduced from
interaction (sequence of collaboration diagrams) or from activity or state machine
diagrams.
Create behavior. At this point, decisions regarding what is directly related to the
distributed system and what is not must be taken. In the case of SAS, a very specific
subsystem, Communications, was devoted to ICa ORB related entities. Inside this
package the elements mapped for the IEC-61850 to CORBA have been identified.
183
These elements are the ones which either are accessed by the SAS subsystem
functions or that access the SAS functions under remote client invocations. In other
words, the Communications Service asset of the communications subsystem
implements the abstract entities of the IEC-61850 standard that describe the
architecture of an Intelligent Electronic Device for SAS. As shown in Figure 69, an
IED is an aggregation of logical nodes which can be of different types.
Figure 69: The ACSI implementation from IEC-61850
As can be easily understood, different device models will support different functions
for SAS. Even the same device models support different SAS functions in the same
substation. For this reason, the devices are configured by means of an XML file and a
configuration tool. This tool specifies the number and type of the logical nodes of the
physical hardware device. There are many different types of logical nodes that
collaborate to perform SAS functions. Examples of these logical nodes are circuit
breaker controllers, switchgear interlocking managers, station-level alarm and event
184
managers, etc. Depending of the functions assigned to the device, different ICa objects
will be created at start-up when the configuration file for the device is loaded.
At the same time, ICa solves several problems of the real-time properties of the SAS
domain. These are the properties related to storage, processing and communication
functions. This functionality also depends on the features available in the devices
where ICa ORB and the SAS framework are embedded so they are easily
configurable. These features are what were previously called variability points of ICa
ORB. Behavior is created with ICa by deciding what ICa ORB communication
resources can be used by SAS applications, what level of concurrency is needed for
the incoming requests to a device and by deciding the available storage space for
incoming requests. It is at the implementation level, with the concrete devices and
resources in them, when the configuration of the variability points of ICa ORB take
greatest importance, although nothing precludes determining these parameters at this
stage for all device models.
SAS Control Objects Framework implementation. The last phase of SAS domain
design is that of the framework implementation. In the case of SAS, framework
implementation is mainly related to the construction of the different logical nodes that
make up SAS systems according to the IEC-61850. Among the problems of
implementation of the object framework for the domain are those of making a portable
development for all the devices in which the objects must run (e.g. different models of
PLCs) and those related to implementing the different configurations which have been
devised for the framework (e.g. optional features).
6.6 CORBA Control Systems Implementation
At the end, the objective of domain engineering is to better construct the systems of a
certain domain. If done correctly, domain engineering has obtained a set of assets and
a framework that can be reused many times over with better guarantees for the
systems’ quality attributes compared to those obtained by building systems from
scratch.
With the use of ICa domain analysis, design and implementation the core of
subsystems, assets and objects for a domain can be transformed into specific running
distributed control systems by means of application engineering. At last, the
implementation of a CORBA control system with the ICa methodology for a domain
is largely a matter of selecting the required features from the domain framework and
185
of configuring its variability points. The objective of the methodology is producing as
few ad-hoc implementation as possible.
6.6.1
Steps of CORBA Control Systems Implementation
with ICa
CORBA Control Systems implementation with ICa is related to application
engineering but in this case, after ICa domain engineering, application implementation
is reduced to a minimum. The following items present the steps that direct application
implementation with the ICa methodology.
1. Identify the requirements of the system. The first phase or step to carry out
consists in finding out the requirements of the system to be developed.
Finding the requirements is important because it helps to define the
functionality of the system. Notice that if, for example, requirements are
found which are not supported by the framework developed during domain
engineering it might be time to perform more domain engineering in order to
add new features to the domain framework. In practice, this means that the
domain framework is built in an iterative process while more and more
systems are built and more knowledge is obtained.
2. Match system requirements to domain framework features. When a
framework for a domain of control systems was built, there were features
already supported by ICa. In the same manner, when building a specific
control system there will be many requirements which will have already been
addressed by the subsystems, assets and objects of the control framework and
by the broker itself.
3. Identify missing features. The framework developed during the domain
engineering effort must try to thoroughly address the problems of the domain.
However, there are situations in which this is not possible. For instance, there
might be a requirement to use tools from a specific vendor at the advanced
control network level for political reasons or, modestly, the object framework
engineer did not envisage some features. In this case, the missing or additional
features must be identified in order to implement them in the final system.
4. Perform application engineering. Finally, it is the time of the application
engineering effort. Application engineering is divided in two efforts. The first
one is basically an effort of deciding what subsystem, assets and objects from
the domain framework to include in the control system being built and of
configuring them. The second one consists in the development of the
186
additional functionality not built into the object framework for the domain.
For this latter effort, a traditional development process can be followed.
Figure 70 shows a schema of the steps involved in constructing CORBA Control
Systems implementations based on the ICa object request broker and on an object
framework specific for a control distributed domain.
Identify
system
requirements
Match
requirements
and features
Identify
missing
features
Application
engineering
DocumentList
FileMgr
Document
add( )
delete( )
fetchDoc( )
sortByName( )
name : int
docid : int
numField : int
get( )
open( )
close( )
read( )
sortFileList( )
create( )
fillDocument( )
FileList
fList
add( )
delete( )
read() fill the
code..
1
rep
File
Repository
(from Persistence)
read( )
GrpFile
name : char * = 0
read( )
open( )
create( )
fillFile( )
readDoc( )
readFile( )
UI
MFC
DocumentApp
RogueWave
Persistence
global
Application engineering and
framework configuration
Steps of CORBA Control Systems Implementation
Figure 70: Phases of CORBA Control Systems implementation
6.6.2
Example of ICa Control Implementation for a SAS
Application
In the examples of the previous sections it was explained how a framework of control
objects for the domain of SAS applications was built. In this section, an example of
the benefits of making application engineering for a particular SAS system with the
help of that framework and of ICa ORB is presented. The section contains the steps of
ICm implementation particularized for a SAS application.
Identify the requirements of the application. The SAS distributed control
application was built to control one high-voltage line of the Red Eléctrica Española
400 kV open-air park substation in Villaviciosa de Odón near Madrid. Figure 71
shows the electrical 1-wire diagram of the substation with the line to control inside a
green marquee. The line is composed by three circuit breakers, two line switches, four
position switches, two bar switches, four measuring current transformers, two line
187
voltage measuring capacitive transformers, two blocking coils and an associated lane
control room. Among the requirements of the SAS application are the following:
•
A group of requirements supported by deeply embedded devices at the bay level:
• Physical device self-checking.
• Control functions.
• Event management.
• Protection functions.
• A group of requirements to be supported by substation level devices (including
substation computers and the operator interface tool):
• Still-alive image acquisition.
• Network management.
• Time synchronisation.
• Software and configuration management.
• Operative control mode of Logical Nodes.
• Setting.
• Alarm management.
• Station-wide interlocking.
• Automatic switching sequence.
• Management, storage and display of the still-alive images.
• HMI to display the substation window, the bay window and the maintenance
window.
• A group of requirements supported by the System Configuration tool, working offline:
• Generation of configuration files.
• Load and storage of SCL files, IEDs capabilities, layout of a power substation
and layout of substation primary equipment.
• Graphical display of substations and bays layout with possibility to configure
the line diagrams.
• Activation / deactivation / configuration of Logical Nodes and functionality.
• The application has real-time requirements for all control related devices except
for the operator interface tool, which is developed in Java language (the SAS
framework is developed in C++) and uses the Borland’s Visibroker commercial
ORB for communications.
188
In summary, the SAS application for the lane contains two IEDs placed in the lane
control room which assume control and protection functions. There is also a substation
computer placed at the substation control room which runs the HMI and is capable of
undertaking some control functions. There is also a configuration terminal for the SAS
application on a portable computer and all the systems are hooked up by a fiber optics
network. Access to the lane control room is controlled by the HMI with direct digital
live-video feed.
Figure 71: The REE substation in Villaviciosa de Odón
Match system requirements to domain framework features. The second step
consists of matching the requirements of the SAS system for the REE substation
against those supported by the framework developed on top of ICa for SAS systems.
As the developed framework is built on the concepts of the IEC-61850 standard, it
results that the matching is performed by identifying the logical nodes of the
framework that support the requirements. The SAS framework addresses all the
control and protection requirements for the SAS application by the use of logical
189
nodes such as circuit breakers, interlocking managers, etc. What this means is that the
control part of the application is solved by configuration of the framework for the lane
to be controlled.
Regarding real-time features of the distributed control system for SAS, some
requirements are directly solved by ICa ORB. The system can automatically be
configured from configuration files about a number of features controlled by ICa
ORB, (e.g. the number and priority of threads running in the IEDs or the use of a
multiplexed connection for an IED). Also the system in the IEDs was configured
according a client propagated priority model and the use of invocation timeouts was
used to control error conditions. Further, ICa solves the problem of integration with
external systems by means of ORB interoperability. The HMI is a Java application
that uses an ORB from a different vendor. There is a direct integration between the
two ORBs by the use of the common interoperability protocol IIOP. Further, the Java
ORB could be cropped as remote method invocations in Java use IIOP as the
underlying protocol. The use of Visibroker was decided to ease the work at the HMI
side.
Identify missing features. It is very difficult to directly implement a complex
distributed control system just by the simple configuration of a framework. When
moving from the control domain concepts to the real-world applications there are
many situations that will have been overlooked. Some of them will be worth of
refining the built framework as will appear in more systems so they must be
incorporated to the domain model while others simply are specific to the system under
work. In the case of the REE substation, there were the specific requirements of
supporting access to the lane control room by a direct video feed and those of
developing the HMI and system configuration tool interfaces. These are the missing
requirements for the SAS application. Of course, the configuration tool and the HMI
for SAS are worth an effort of domain engineering as can be reused in many systems
but, in this case, were not considered part of the developed SAS framework based on
the IEC-61850 standard.
Perform application engineering. For the SAS system, application engineering is
performed by building the appropriate configuration files for the substation devices.
The framework objects that control the substation gear are built on system start-up
from the configuration files. The additional work is that related to the development of
the off-line configuration tool and that of the human-machine interface. It is important
to notice that even if the framework does not support the HMI it greatly reduces its
development cycle as the SAS framework on top of ICa ORB already gives the HMI a
190
domain layer (that of the SAS framework objects) that is employed together with the
Java ORB in the development of the HMI.
6.7 Pattern and Architecture Exploitation
ICa and its methodology provide a software framework into which CORBA Control
Systems can be organized and developed. In the development of ICa, software
engineering and architecture principles that eased the work of implementing the
complex pieces of ICa software have been followed. Thus, the ICa ORB is composed
of many subsystems that interface to each other and intensive use of Pattern Oriented
Software Architecture (POSA) [Bus 96] solutions has been employed. The purpose of
all these efforts has always been to base development on sound techniques so the final
quality of the software will be good. However, even if performance is among the
initial requirements for ICa ORB, there is always the perception that too much
structure into systems results in a loss of performance compared to what we might call
“dirty code hacks”.
First, it should be noticed that, in our experience, there is no way to produce, maintain
and evolve a large production system which is built around unstructured code and
blurrily partitioned subsystems. Secondly, there is the fact that it is not possible to
develop long-life systems without being involved in an effort of domain engineering.
A system that must survive many years has the requirement to evolve and evolution is
tremendously facilitated by domain engineering. The problem is that of modeling the
objects of a domain but, once this is achieved, the benefits are enormous.
Once domain engineering has been performed for a family of systems and the objects
of the domain model have been identified it is possible to query about optimizations or
performance. The idea to follow is something like “first structure then performance”.
Nevertheless, it is also good to recognize that unpleasantly optimized code or dirty
hacks are sometimes necessary. The best way to do this is to put the correct objects in
the domain model so optimizations are enclosed inside objects and do not affect the
rest of the system from the development point of view. For instance, ICa ORB
incorporates the concept of thread in its model. Initially the hopes of portable
development were concentrated in the POSIX standard which should have avoided
many problems of implementation. In practice, reality is that not every operating
system implements the POSIX standard, it is partially implemented or simply it
happens that the native services of the operating system perform better than their
POSIX equivalents. As an example, this happens with the VxWorks operating system
where threads are just tasks and where synchronization object acquisition performs
191
better when native operating system calls are used. ICa ORB has a thread object
model which is optimized for each operating system. This effort of domain
engineering results in a sustainable and evolvable improvement of performance. The
idea is to conceptually firewall either the unstructured parts of ICa (because they are
too close to a particular operating system implementation) or the modeled domain
from other subsystems.
As always, the pragmatic approach adopted in this work is shown with an example.
Following the development of the SAS control framework and its exemplar system for
the Villaviciosa de Odón REE substation some performance experiments were carried
out during the works of the DOTS project (see Section 7.1.2). The experiment
physical layout in the REE substation is shown in Figure 72.
OIT
IHMI0
3'
5'
1
6
IED2
IED1
XCBR20
XSWI25
2
5
3
CILO0
4
Figure 72: Physical layout for interlocking tests
The figure shows a simplified schema of two intelligent electronic devices placed in a
lane control room of the substation (IED1 and IED2). IED1 contains, among others, a
logical node (CILO0) which is devoted to the interlocking control of subsystem
elements. In a substation some elements must be in certain controlled states before
other operations on some elements can be performed. For instance, a circuit breaker
cannot be closed if its corresponding switch is open. The process of operating the
substation organs and of checking their state is called interlocking. IED2 contains a
logical node representing a switch (XSWI25) and a logical node representing a circuit
breaker (XCBR20). The box in the upper right part of the figure labeled OIT is the
Operator Interface Tool running in the substation computer of the substation control
room and implements a logical node (IHMI0) designed for user interface purposes.
192
The OIT runs the commercial ORB Visibroker whereas the IEDs with more stringent
real-time requirements run ICa ORB. The orange box at the upper left corner of the
figure shows two substation elements that must be operated in an interlocked way. The
one on the left is a switch and the one on the right is a circuit breaker. For these
experiments, the IEDs were configured with a supervision scan rate of analog and
digital signals of 100 ms.
The numbered yellow circles represent the sequence of actions to perform an
operation. First, the operator commanding the HMI tool issues a command to the
appropriate circuit breaker in IED2. The logical node representing the circuit breaker
queries its associated interlocking control node (CILO0) on IED1. The CILO0 logical
node queries then the associated switch about its state. This information is used then to
inform the circuit breaker on the actions to perform. If the operation is rejected the
user interface is updated (Figure 73). If the operation is allowed the circuit breaker
logical node (XCBR20) closes the organ and the operator interface tool is updated
(Figure 74).
Case 1: operation inhibited
º
OIT. IHMI0
IED2. XCBR20
t1
t2
t3
t4
t5
t6
→
XCBR.set_ctlVal
IED1. CILO0
CILO.get_stVal
IED2. XSWI25
→
XSWI.get_stVal
→
opened
←
←
←
Ethernet Frames - GIOP (time in microseconds)
us
1
2
3
20.335.662
45.553.513
52.144.590
t1
20.406.110
45.614.671
52.203.864
t2
20.473.052
45.682.335
52.271.569
t3
20.610.142
45.832.117
52.416.208
t4
20.635.061
45.864.838
52.443.386
t5
20.659.827
45.888.113
52.467.953
t6
70.448
61.158
59.274
t1-t2
66.942
67.664
67.705
t2-t3
137.090
149.782
144.639
t3-t4
24.919
32.721
27.178
t4-t5
24.766
23.275
24.567
t5-t6
324.165
334.600
323.363
t1-t6
disabled
rejected
4
58.703.645
58.768.336
58.832.201
58.968.215
58.990.098
59.012.211
64.691
63.865
136.014
21.883
22.113
308.566
5
64.313.420
64.374.433
64.442.697
64.582.785
64.604.270
64.626.445
61.013
68.264
140.088
21.485
22.175
313.025
6
74.245.842
74.314.190
74.383.529
74.523.270
74.544.635
74.566.733
68.348
69.339
139.741
21.365
22.098
320.891
7
79.532.090
79.594.563
79.666.425
79.804.676
79.826.270
79.852.838
62.473
71.862
138.251
21.594
26.568
320.748
8
95.386.618
95.449.168
95.518.615
95.657.546
95.679.531
95.701.697
62.550
69.447
138.931
21.985
22.166
315.079
9
97.130.014
97.190.656
97.262.690
97.402.515
97.424.028
97.446.218
60.642
72.034
139.825
21.513
22.190
316.204
10 avg.
max.
106.980.377
107.041.147
107.110.436
107.250.338
107.272.837
107.295.188
60.770
63.137
70.448
69.289
68.641
72.034
139.902
140.426 149.782
22.499
23.714
32.721
22.351
23.227
26.568
314.811
319.145 334.600
Figure 73: Interlocking operation rejected
The figure above shows the case when the closing operation of the circuit breaker is
rejected because the switch is still in the open state. Time measures between during
the flow of requests among the logical nodes are denoted t1 to t6. Average and worst
193
case times are presented in the right part of the table. Notice that the biggest lapse of
time occurs in the t3-t4 interval. This is because the switch logical node has to
physically connect by the wiring to the switch element in the substation and get its
state. Afterwards, the switch logical node returns the switch state to the interlocking
logical node and from there to the circuit breaker and HMI.
Figure 74 shows the case in which the operation is not inhibited and thus the circuit
breaker can be closed.
Case 2: operation allowed
OIT. IHMI0
IED2. XCBR20
t1
t2
t3
t4
t5
XCBR.set_ctlVal
→
t6
←
CILO.get_stVal
IED2. XSWI25
→
XSWI.get_stVal
→
closed
←
←
close
Ethernet Frames - GIOP
us
1
2.499.720
t1
2.563.527
t2
2.631.717
t3
2.767.646
t4
2.789.230
t5
2.892.260
t6
63.807
t1-t2
68.190
t2-t3
135.929
t3-t4
21.584
t4-t5
103.030
t5-t6
392.540
t1-t6
IED1. CILO0
(time in microseconds)
2
3
5.365.568
7.835.573
5.429.650
7.900.292
5.493.665
7.966.715
5.630.963
8.104.121
5.652.693
8.128.383
5.762.258
8.232.135
64.082
64.719
64.015
66.423
137.298
137.406
21.730
24.262
109.565
103.752
396.690
396.562
enabled
IED2. SD1
→
accepted
4
9.296.164
9.361.910
9.427.648
9.570.560
9.596.547
9.698.891
65.746
65.738
142.912
25.987
102.344
402.727
5
15.425.039
15.487.889
15.565.181
15.704.305
15.726.077
15.829.857
62.850
77.292
139.124
21.772
103.780
404.818
6
19.646.169
19.709.996
19.784.617
19.924.838
19.956.273
20.064.049
63.827
74.621
140.221
31.435
107.776
417.880
7
21.981.022
22.046.448
22.112.679
22.255.135
22.276.619
22.386.585
65.426
66.231
142.456
21.484
109.966
405.563
8
24.083.936
24.149.677
24.218.979
24.362.593
24.387.501
24.492.481
65.741
69.302
143.614
24.908
104.980
408.545
9
28.529.510
28.593.594
28.661.929
28.803.099
28.824.590
28.930.021
64.084
68.335
141.170
21.491
105.431
400.511
10 avg.
max.
32.992.953
33.060.292
33.125.986
33.269.912
33.295.873
33.405.228
67.339
64.762
67.339
65.694
77.292
68.584
143.926
140.406 143.926
25.961
31.435
24.061
109.355
105.998 109.966
412.275
403.811 417.880
Figure 74: Interlocking operation permitted
In this case, the operation is permitted and the circuit breaker is closed. Notice that the
total time to perform the operation is a little larger than in the previous case as the
circuit breaker has to be physically closed. This can be seen in the t5-t6 interval which
is now larger than in the previous case.
About the validity of the result and the performance of ICa and the developed SAS
framework it must be said that the requirement for operator operations was of about 1
s. This means that the results obtained are very good. Further, when compared to other
substation control systems the figures found for this type of operations are similar to
the ones achieved in the experiments. This must be considered, taking into account
194
that the SAS implementation is a first implementation and has not been optimized as
other commercial substation control systems. These results validate the domain
engineering approach taken for CORBA Control Systems with ICa, its broker and
methodology.
195
Chapter 7
Exemplar CORBA Control Systems
197
7.1 Summary List of Funding for this Work
The work in this Thesis has been developed under the umbrella of several
international projects funded by the European Commission during the ESPRIT and
IST programmes. The following is a list of projects that have directly funded the
research. There are other projects in which the results of this work have been applied
but that are not listed here.
7.1.1
IST-2001-37652 Hard Real-Time CORBA (HRTC)
The focus of this project was that of distributed control applications. This type of
applications is usually represented by a distributed information system that closes one
or more control loops to keep several interacting target systems in a controlled state.
The temporal behavior of the control system is critical in this type of applications due
to the dynamic effects that arise from delays or jitter produced from the software /
hardware execution path needed for control algorithm calculations.
Distributed Control systems are software-intensive applications that are becoming
extremely complex as more and more functionality is being required from them.
Complexity as a real engineering challenge can be successfully addressed by
distributed object technology. As already known and presented in this Thesis, one of
the leading technologies regarding distributed objects is the OMG’s object request
brokering model proposed by the CORBA specification. While CORBA specifications
do address real-time issues they deal only with soft real-time systems and that, simply,
is not enough for certain types of distributed systems (namely controllers) where
timing properties are critical. The construction of distributed CORBA systems where
time-related properties are critical was the objective of HRTC.
Concretely, the central objectives of HRTC were (i) the analysis and identification of
hard-real time requirements posed by CORBA-based distributed control systems, and
to develop theory / methodology for hard-real time CORBA applications, (ii) the
enhancement of CORBA specifications with corresponding interfaces in order to build
distributed control systems that have real-time requirements with hard timing
constraints, and (iii) the implementation of CORBA-pluggable real-time protocols in
the ICa ORB for experimental tests.
The requirements were developed based on partners' experience and on project
specific experiments in two testbeds (one in robot control and one in continuous
process control). They were used to launch a specification process inside the OMG by
199
means of the preparation of a RFI (Request for Information), a RFP (Request for
Proposals) and by fostering the collaboration with other OMG groups in the
elaboration of a proposal in response to the RFP.
The participant partners in HRTC were mainly academic with the exception of
SCILabs Ingenieros from Spain. Three universities provided their expertise in this
project, UPM from Spain in the field of distributed objects, the University of Lund
from Sweden as expert in control theory of networked systems and, the Technical
University of Vienna from Austria with its knowledge of time-triggered architectures.
7.1.2
IST-1999-10251 Distributed
Systems and Networks (DOTS)
Object
Telecontrol
The main objective of the DOTS project was to establish an open software model built
upon real-time distributed object technologies and emergent telecontrol standards to
allow the optimum exploitation of the interoperation capabilities of devices (Intelligent
Electronic Devices) and systems in the distributed context of an electric power grid.
The project addressed outstanding problems in this context, and its results are of
application in any other industrial control system. Real-time and Minimum CORBA,
and the, at the time of the project, emerging IEC-61850 standard for control systems in
electrical substations were the supporting technologies of DOTS.
In DOTS, the promotion of real-time distributed object technologies for teleoperation
of control systems and networks and, the contribution to standardization organizations
like IEC were among the core project ideas. Also, in a pragmatic approach,
development of tools, middleware and the effective operation of a pilot telecontrol
system was carried out within the project. Thus, development helped us to
demonstrate the validity of the DOTS approach when facing real-world conditions.
Several objectives were pursued during the project. Among them, there were the
following:
• Definition of a general model for distributed telecontrol networks at electric power
grids. This model considered CORBA performance versus the hard real-time
constraints imposed by the IEC communications model, specifically identifying
QoS (Quality of Service) requirements of the station elements.
• Mapping of the IEC-61850 Abstract Communications Service Interface (ACSI)
[IEC 01b] to CORBA. Both the general model definition and the mapping
specification documents were submitted to the involved IEC and the OMG (Object
200
•
•
•
•
•
Management Group) Technical Committees, in order to contribute to the
promotion of IEC-61850 and Real-time and Minimum CORBA standards.
Development of a substation configuration tool for the interoperation of devices at
the configuration level, according to the IEC-61850 configuration language
specification.
Development of an operator interface tool for accessing the objects embedded in
distributed and heterogeneous devices located at a power substation. This tool
made use of commercial non-real-time CORBA middleware and Java language
running in a PC platform and interoperated with the real-time ICa ORB embedded
in the RTUs.
Implementation of base software, mainly selected IEC-61850 objects and
functions, to constitute a development framework on top of which
communications networks and industrial control systems can be implemented. The
IEC standard functions were represented as ICa CORBA objects so they were
accessible by CORBA enabled applications like the operator interface tool.
Adaptation of an existing real-time ORB (ICa) to be embedded into a RTU at the
substation level.
Integration of the middleware, software and tools developed within the project into
a pilot system located at a power substation belonging to Red Eléctrica Española to
validate the selected approach.
The departure point of project was the worldwide recognized necessity to ease the
interoperation of systems, networks and devices in electricity distribution installations.
The necessity arises from the overwhelming amount of protocols and proprietary
interfaces present in this context. This necessity was targeted by two approaches in the
context of power substations that were in draft status at the time the project began; the
American UCA 2.0 architecture, and the IEC-61850 standard. At that time, to the
Consortium’s knowledge, and due to the draft status of these standards, no IEC-61850 or
UCA 2.0 compliant system had been implemented nor installed. In fact, the pilot system
installed during the project was the first CORBA-based running implementation of the
IEC-61850.
At the same time of the project, preparation of the Real-time CORBA specification,
version 1.0, had just finished and several commercial products for real-time distributed
objects technology had been developed under CORBA 2.2 specification. However, it
was not usual the integration of this middleware in a real-time environment as hard as
the control of embedded devices in a power substation distributed network. Although
the project initially targeted the specific context of power substations, it was also aimed
to overcome general and outstanding problems that arise in communications networks
201
and distributed systems for industrial control, always related to the integration and
interoperation of heterogeneous devices, networks and systems of different
manufacturers.
Distributed telecontrol networks are composed of a wide variety of systems belonging to
different categories, depending on their functionality, location and performance
requirements: real-time IEDs (intelligent electronic devices) and deeply embedded RTUs
(remote terminal units), maintenance and operation terminals and, at the top level,
Supervisory Control and Data Acquisition (SCADA) systems located at control centres
of utility companies. Communication among all these devices is achieved through the
available links at the field site. DOTS provided a clean open way for interoperation
among all of them by providing an architecture with the mixed fundamentals of IEC61850 and embedded real-time CORBA.
Partners from four European countries were represented in the DOTS project. ELIOP,
UPM, REE and SCILabs Ingenieros from Spain, UNINOVA from Portugal, DECANGETRONICS from France and INNOVA from Italy.
7.1.3
ESPRIT Project 22130 – Distributed Information
Technology for Strategic Multiobjective Process Control
(DIXIT)
The project aimed at producing a software and methodological environment targeted at
preventing and solving control problems at different layers of intelligent control
(operational, tactical and strategic). The strategic situations to deal with focused on
safety, environmental and maintenance issues and the software environment consisted of
AI problem solving distributed components integrated by means of a flexible and
distributed architecture (ICa). Knowledge engineering techniques and software
engineering methods, including object oriented technologies were used and combined
with distributed architecture paradigms in order to produce cheaper and more efficient
software.
A basic objective was to provide an industrial competitiveness advantage by improving
automation in a reduced cost approach. Cost was reduced by the construction of generic
software components aimed at addressing different types of problems at different control
layers, supported and integrated by a flexible distributed-objects architecture, and the
definition of a methodology. This was the commitment of the DIXIT project partners.
202
The structure of most distributed control systems is hierarchical and multilayered, similar
to a pyramid, where abstraction as well as control mechanisms complexity increases in
higher layers. DIXIT dealt with problems and situations included in the intelligent control
layers; operational, tactical and strategic. In particular, it focused on the strategic level,
especially on environmental, safety and maintenance aspects considering them together
with other global objectives of the plant, such as production, quality, etc. All these
objectives constituted a constraint network where priorities and weights were assigned
depending on the problem at hand.
From the technical point of view, the DIXIT final product was composed of different
software modules: user interface components, problem solving components and
architectural components. The problem solving modules performed tasks such as failure
detection, diagnosis, monitoring, tactic creation and recommendation, sensor validation,
forecasting and trending, alarm and emergencies handling, prevention, optimisation, etc.
The technologies used spanned rule-based systems, neural networks, fuzzy logic,
optimisation techniques, etc. In order to support system ubiquity, the DIXIT architectural
components combined the blackboard paradigm (already proven as a successful approach
for intelligent applications in a previous project called HINT) with other paradigms
related to distribution, such as an agent architecture based on the CORBA specification
(ICa).
The consortium was formed by five technical developer organisations (IIC, acting as coordinator, RH&H, DaCapo, UPM and Euristic Systemes) and two users (Repsol Quimica
and Lafarge Ciments). Four European countries were represented in the consortium.
7.2 Exemplar Applications of ICa
ICa has been tested in a wide span of distributed control applications. The deployed
systems vary from huge plant-wide systems running at all levels of the control
hierarchy (from capture of data to control strategy definition) to embedded systems
designed for hard real-time. The following are some examples of the applications of
ICa in the control world.
203
7.2.1
Process Control Testbed
This application was developed in the scope of the HRTC project at ASLab facilities
in E.T.S.I.I. The main objective of the Process Control Testbed (PCT) application was
to identify (mainly hard real time) requirements for distributed control systems and to
perform experiments in conditions of system heterogeneity and integration of legacy
and proprietary applications. Experiments were done using conventional IIOP and the
ICa real-time protocol plug-in developed for TTP. Figure 75 shows the hardware
setup for the process control application.
Figure 75: Hardware setup for the process control
testbed
One of the general requirements for this process control system was that it should be
representative of a real plant process control system. The process control system
mimicked a plant control system, what can be seen as a (possibly hybrid) redundant
network where the nodes are monitoring and control elements. Therefore, the process
control system was essentially a redundant network where system components are
connected. The hybrid characteristic of current industrial control systems, with distinct
networks at different levels, was intentionally eliminated to test the viability of a flat
network in control environments. On the other hand, since a conventional (non hard
204
real-time, TCP/IP) and a specialized (HRTP, TTP/C) networks were being compared,
the heterogeneous features of a conventional DCS were recovered just by joining both
networks in further experiments.
The designed process control demonstrator was able to comprehend the functionality
of both present and future process plant control systems. The idea was to build such a
control system using CORBA components and to check whether it was possible to:
1. Perform the tasks that current systems usually do.
2. Accomplish the tasks that future systems are expected to achieve.
A progressive approach to the construction of the process control application was
devised in order to test features incrementally. This progression was concretized in a
series of scenarios (different network configurations) where a set of experiments was
performed. The planned / executed experiments were the following.
1.
2.
3.
4.
5.
6.
7.
Single control loop.
Legacy system integration.
Simulation components integration (not performed).
Sequence of events generation.
Traffic capacity test.
Concurrent access.
Response to faults (not performed).
These experiments were run on the conventional (Ethernet) and specialized networks
(TTP). The final experiment (Figure 76) combined both networks through a bridge.
From the batch of tests conducted on the process control platform it was concluded
that the integration of CORBA in control loops did not pose significant problems to
developers. CORBA is an alternative for process control not only were OPC operates
but also at lower levels (field level). The main problem for this to happen is the lack of
openness of existing DCSs which impedes the access to system functionalities from
the CORBA layer.
205
Figure 76: The process control demonstrator
7.2.2
Distributed Robot Control
The testbed for distributed robot control was also developed in the frame of the HRTC
project at the Automatic Control Department of the University of Lund in Sweden.
The objective of the robot control demonstrator was to clarify the motivation for hard
real-time CORBA. This experiment aimed to illustrate the timing deficiencies in
206
distributed computing for robot control between a regular ORB and a hard real-time
CORBA ORB, namely ICa ORB.
Robot control is a good testbed because they are often required to be very flexible so a
component-oriented design based in CORBA objects could be a good match for this
type of systems. For the experiments setup advantage was taken of the available
facilities of Lund’s University. The primary system included an ABB IRB2000
manipulator, which was completely reconfigured by the Swedish team to permit
control experiments. The original control computers were removed, hardware
interfaces were added, and their own control computers were connected thus forming
a completely open controller, including servo control down to the torque control of
AC motors, which requires several kHz sampling frequency for each of the six joints.
Figure 77 shows the setup for robot experiments.
Camera
Actuator (motor)
Windows PC
ETRAX
Internet
Sensor (resolver)
• Linux/RTAI
ETRAX
• Linux/RTAI
PPC * 2: Linux/RTAI
& STORK
Switch
GlobeThrottle
• Linux PC
Controller
Figure 77: Robot control testbed setup
The basic purpose of the robot control experiment was to catch a thrown object (a
ball) using the 6-DOF industrial robot. The ball trajectory was estimated by means of
a stereo vision system and a prediction for the catch point was obtained as input for
the robot. The experiment was originally developed (without CORBA hard real-time
extensions) as a PhD project at the Department of Automatic Control in Lund. Figure
78 shows a pitch of the ball to the robot. Two wall-mounted cameras provide the
stereoscopic vision input for calculation of ball trajectory.
207
Figure 78: The Irb-2000 robot ready to catch
the thrown ball. Notice the two cameras on
the wall.
The hard real-time core of the system depicted in Figure 77 is formed by the four
nodes Sensor, Controller, Actuator and the Globe Throttle PC configured to schedule
hard real-time network traffic in RT-Ethernet. The Globe Throttle and the switch
guarantee that an upper bound for hard real-time communications in the network can
be controlled. This is achieved by limiting the traffic bandwidth that each node is able
to request from the network. Technically this is called throttling. For this environment,
ICa ORB was ported to the RTAI-Linux platform so it could be used to wrap the
control APIs of the testbed nodes.
7.2.3
Substation Telecontrol Application
The DOTS project was focused towards the development of open control system models
based on standards and on an open software architecture built upon real-time distributedobject software technologies.
Among the most important results of DOTS there was the implementation model of the
IEC-61850 standard (Figure 79), designed for communication networks and systems in
208
substations. Interoperability among multi-vendor control devices was achieved by
mapping the IEC standard to a real-time CORBA middleware, ICa ORB. The mapping to
CORBA was submitted in the framework of the project for inclusion as part of the IEC61850 standard. The standard specifies a collection of elementary services that can be
performed by substation automation systems or SAS. These services such as breaker
operation, interlocking, system monitorisation, etc. are offered in the standard by artifacts
called Logical Nodes that are distributed physically in different processors and mapped in
DOTS to CORBA objects.
Figure 79: IEC-61850 standard perspective on
function provision based on logical nodes
To achieve global behavior in the substation automation system, the distributed logical
nodes collaborate to perform Logical Functions. To check the validity of the DOTS
approach, a pilot system was developed and installed at a substation of Red Eléctrica
Española in Villaviciosa de Odón (Madrid), incorporating the software and tools
developed within the project. These tools were the following:
• A deeply embedded device (IED or Phasor Measurement Unit, to acquire
three voltage and three current measures and to calculate phasors and
frequency from them) at the bay level supporting the following IEC-61850
functions:
• self-checking,
• configuration and maintenance,
• control,
• protection
• and distributed process automation.
209
• A RTU at the substation level, that included the following IEC-61850
functions:
• network management,
• configuration and maintenance,
• alarms and events management,
• and station interlocking.
• Another RTU that included extended capabilities, like still-alive image
acquisition, compression and transmission for surveillance purposes.
• An Operator Interface tool for local or remote inspection and interaction with
objects distributed in the power substation.
• A System Configurator tool, working off-line, to provide interoperability at
the substation configuration level.
• ICa ORB, the embedded real-time CORBA middleware for communications
among the existing distributed objects, using serial and Ethernet links.
Through the pilot system (Figure 80) operation, the following key points were
demonstrated:
• Adequacy of the model for communications networks and systems in power
substations and grids.
• The flexibility of the power substation objects was not constrained by the
communication layout (CORBA versus legacy protocols). Extended functions, like
surveillance, not envisaged by current telecontrol protocols, were smoothly
integrated into the pilot system.
• Dynamic reallocation of the SAS related objects, as well as balance of the
processing load among the pilot system RTUs.
• Enhanced visibility of the telecontrol objects from operator interfaces or SCADAs
located at remote sites, e.g. at a control centre.
As a conclusion the project demonstrated that the implementation of IEC-61850 over a
CORBA middleware offers extremely good perspectives for complex systems
construction and deployment, tasks that are quite more complex using other types of
technology.
210
The advantages obtained for maintenance must not be disregarded. CORBA enables
running application analysis and monitorisation and simplifies system evolution.
Changing platforms (hardware or operating systems) are not problematic for CORBA
based applications as it is in CORBA’s nature to provide transparency across
heterogeneous system boundaries.
Non Real-time
Configuration
Terminal
Operator
Terminal
Station
Network
CORBA
Monitor
CORBA
gateway
IED-1
Process
Network
CORBA RTU
IED-2
Real-time
Figure 80: Setup of the DOTS pilot system
7.2.4
HYDRA-Vision
Hydra-Vision relays real-time digital video to operators distributed in a WAN
environment. The Hydra-Vision system has been developed upon the ICa ORB
framework to provide control-room engineers with real-time video feed from remotely
operated power-generation plants. The system is composed of a collection of nodes that
generate video streams (usually video cameras in plants) and another collection of nodes
that are clients of these streams. These last nodes can be operation stations, supervision
stations, security offices, etc.
Video and audio streams are generated using remotely-controlled video cameras and
microphones and compressed using real-time MPEG compressors based on hardware.
These systems are organized into zones that are controlled by a single system. The
equipment (controller, MPEG compressor, multiplexers, etc.) corresponding to a zone
receives the name of a Local Control Center (CCL). All these elements are built as ICa
agents (CORBA objects), offering CORBA services to other agents in an open
architecture.
211
Hydra-Vision employs ATM (Asynchronous Transfer Mode) broadband networking as
the base communications infrastructure but, for deployment reasons, IP over ATM was
used for all network APIs instead of using native ATM APIs that offered better control
over resources [Seckin 98]. The classic network service on the Internet (based on IP) is a
best-effort service, where packets from a source are sent to a destination, without
guarantees of delivery or timing [Borden 95]. Video traffic requires better control over
network resources to guarantee video quality at the client side [Pancha 93]. Hydra-Vision
applications are built on top of two virtual networks:
•
•
The control network handles all service negotiation between Hydra-Vision
agents using ICa technology. This is the network that all Hydra-Vision agents
use to exchange information and service requests with all other agents.
The video network handles all video and audio traffic. This network employs
multicasting to minimize bandwidth use [Ravindran 97], making it possible for
different clients to watch the same video stream concurrently. An authorization
system built in the control network manages access to systems resources.
Remote clients are usually placed in Integrated Control Centers (CCI) and receive MPEG
video feeds, which are decompressed by software and displayed in operator stations.
Video feeds can also be watched in any other station in the network (even if they are
placed in CCLs). The main graphical user interface is shown in Figure 81 and operates
in the three different modes explained below.
Figure 81: Hydra-Vision graphical user interface
212
•
•
•
Local: Used when a user wants to watch a video feed in a workstation of the
CCL where that camera is placed. The user selects the camera, the target subject
and the viewing parameters and the system interacts with all the equipment to
serve the video feed to the client. The user can operate the camera and watch the
video like in a conventional CCTV system.
Remote manual: Used when a user wants to watch a video from a camera in a
different CCL. The operation is exactly the same, except that the distance
between the camera and the viewing screen can be hundreds of kilometers.
Remote automatic: This mode is used for normal operation. Hydra-Vision is
integrated with the DCS used to operate the plant enabling it to respond
automatically to plant events. This mode is used when the DCS signals a special
situation in a plant (for example an alarm triggering).
Hydra-Vision operates semi-autonomously in remote automatic mode; an ICa agent
automatically requests the video system to generate a video stream from a specific target
in the plant to a specific display station. For example, an alarm triggered in a transformer
will target a camera on that transformer, generate the compressed video stream and
display the video stream in the operation station responsible for that alarm. All these
events will happen with a delay of some tenths of a second (mainly due to limited speed
in the camera positioning system).
213
Figure 82: Controlled equipment and Hydra-Vision
network
As the number of concurrent users is not determined in advance and to handle priorities
between users, it is necessary to employ policies that control the response of systems to
remote and / or local requests. This is done to minimize the impact of non-critical video
feeds on critical ones. Criticality is determined by the originating cause and priority level
of the video feed. The system runs over a network of UltraSparc workstations running
Sun Solaris and Pentium based computers running Microsoft Windows NT (to support
MPEG compression boards). Other equipment is also managed by the system as audio
and video multiplexers, video quad equipment and recording facilities for video streams.
Figure 82 shows some of this equipment and a network of CCLs and CCIs.
The potential uses of this type of technology are not restricted to safety applications. This
system can also be employed in:
•
Security: Plants without personnel can be targets for thieves or vandalism. This
system can be used by security personnel in remote sites as a CCTV using bidirectional audio to pretend that there is human presence in the plant.
214
•
•
Maintenance: Experienced maintenance personnel are scarce and by using this
technology it is possible to send less experienced personnel to plants that can
keep contact with the expert using bi-directional audio and video.
Videoconferencing: The homogeneous nature of the system (all nodes share the
same functionality) also makes it possible to use the Hydra-Vision for
videoconferencing.
Broadband integrated DCSs for electric utilities are one of the many applications of
distributed object technology in control systems engineering. The system presented in
this section was deployed in several hydroelectric plants that Unión Fenosa owns in the
province of Guadalajara (Spain).
7.2.5
Teleoperation of a Virtual Robot
Robotic systems work alone no more, they collaborate. Systems are changing from
isolated robots with built-in control to complex systems like manufacturing cells with
multi-robot cooperation which are also integrated with higher-level enterprise systems.
Other complex robotics systems exist, like teleoperated robots with human supervisory
control or humanoid robots. In order to carry out their tasks, different types of sensors
populate the robot work area and the data generated by them must be integrated (e.g.
cameras, force sensors) at different control levels (e.g. path-planning, man-machine
interfaces). The extreme example of these complex robotic systems are the humanoid
robot projects like Cog [Brooks 98], Wabian [Hashimoto 98] and P3 and ASIMO [Inoue
98]. In these projects, components for perception, cognition and mechatronics have to be
integrated into a single system.
To demonstrate the usefulness of ICa also in the field of robotics, the ICa ORB has been
applied to the control of a teleoperated robotics system that includes a force-feedback
joystick master and a simulation of an anthropomorphic robot arm. A virtual robotic arm
has the advantage of freedom of design. Therefore, the virtual robotic arm design was
inspired by the anatomy of the human arm in order to resemble a complex instrument.
The aim of the design is to preserve the mechanical characteristics of the human arm as
much as possible but, at the same time, to enable technical feasibility that allows proving
the advantages of distributed-object system development.
215
The design followed a nature-is-best approach and each articulation complex was
investigated with respect to its biomechanics. Cadaver studies like [Kapandji 84] and
[Helm 92] were useful for this respect although they are rare in literature and the data
varies for each individual. As a result, the model is mainly based on structural data of the
human arm and on the qualitative description of physiological motion. The manipulator
model comprises the shoulder joint, the elbow and forearm joint as well as the wrist
(Figure 83). In the experiments carried out, the 7-DOF robot behaves as a slave while it
is commanded by master joystick with force-feedback.
Figure 83: The simulated anthropomorphic
robot arm
The master joystick has only 6-DOF while the robot has one more degree of freedom.
This problem was resolved by introducing a condition in the system that obliged to keep
the robot forearm in a horizontal position. This means that the simulation of the robot arm
looks like a human arm manipulating objects placed on a table or flat surface. The tool
used to visualize the robot is called KaVis. It is based on Open Inventor software and its
purpose is to visualize the motion of kinematic chains.
For the teleoperation experiments a heterogeneous hardware and operating system
architecture was devised. Heterogeneity is common in teleoperation environments and
one of the points that were to be proved is that CORBA is good for system integration.
The heterogeneous teleoperation environment was built around a PC running Microsoft
216
Windows NT to which the master joystick was linked. The virtual robot was running in
the KaVis visualizer on top of the IRIX operating system in a Silicon Graphics
workstation. To make the experiments more complex, a Sun workstation running Solaris
was used as the link between the master joystick and the slave robot. The mission of the
Sun workstation was to work as controller of the master and slave systems and as a
translator between the master and slave coordinate spaces. The controller software in the
Solaris workstation samples the state of the master Cybernet joystick and updates the
force feedback the joystick user feels. In this software architecture, the master behaves as
a CORBA object which is queried by the controller. The controller is client of the master
but at the same time is a server whose services are requested by the virtual robot which is
a CORBA client. KaVis provides an API for C++ which allowed it to be integrated with
ICa for these experiments.
The experiments not only demonstrated how system development is improved by the use
of ICa but also that multiple robotic slaves that conform to their IDL descriptions could
be controlled. This is shown in Figure 84.
Figure 84: Multiple systems can be teleoperated with a single
IDL interface
217
7.2.6
RISKMAN
RiskMan is a risk manager that was developed as a demonstrator of the DIXIT project
technology. DIXIT addressed the problems of the higher layers of the control pyramid,
mainly the strategic layer. The strategic layer comprises tasks that affect the plant
from a global point of view and commands the lower layers to pursue the defined
strategy. The objectives of the tactical layer can be classified regarding continuity,
maintenance, production quantity, quality, efficiency and safety. The strategic layer
performs a continuous follow-up of these objectives in order to achieve them.
Several strategic objectives were selected to be built into RiskMan. The RiskMan
systems were deployed in the petrochemical complex that Repsol owns in Tarragona.
The complex extends over an area of 150 Hectares and comprises a refinery and a
chemical complex with 9 production plants. The plants are organized into three
different areas according to potential hazard. In a complex like this, the selected
features to test were those related to safety and maintenance.
Regarding safety, the objectives of the system were to prevent the occurrence of
anomalous situations and to assist in dealing with them by making accurate
information available to key personnel. For this, the emergency plan developed by
Repsol to comply with European, Spanish and local regulations was used and an
intelligent system, RiskMan, was developed.
RiskMan prevents and detects dangerous or potentially dangerous situations and
classifies them according to four categories of emergency depending on its impact on
life and property. RiskMan performs its work by using several AI techniques like
neural networks, expert systems or fuzzy technology to supply predictions of system
behavior. If an abnormal situation is detected the emergency plan is automatically
launched providing up to date information to the persons responsible for the execution
of the emergency plan and including a role assignment to each of them. Also external
communications are established in order to inform competent authorities via automatic
phone call, fax or e-mail.
For the maintenance objective, it is also needed, in a system like a petrochemical
complex, to analyze the impact of the maintenance operations in the safety of the plant
before carrying them out. In fact, although some operations may be harmless by
themselves, the combination of several of them in the same geographical area can
yield very harmful effects. This is why a work permits system is needed in the
218
complex. RiskMan manages the maintenance operations being carried out at a given
instant taking into account the personnel and equipment affected by maintenance.
With these constraints, it is able to issue new work permits for a certain area.
The architectural technology that supports distribution in RiskMan is ICa. The ORB
allowed the integration of heterogeneous systems ranging from VAX-based industrial
databases to prediction engines running on Alpha workstations or desktop PCs
showing the RiskMan user interface (Figure 85) to direct and assign people roles for
emergency handling.
Figure 85: RISKMAN user Interface
7.2.7
PIKMAC
PIKMAC (Process Information and Knowledge Modeling for Advanced Control) is
another demonstrator that was developed in the scope of the DIXIT project. Its
objectives, also in the strategic layer, were focused towards production quantity,
quality and maintenance. The demonstrator was deployed in the cement plant that
Lafarge Ciments runs near Nice.
219
For the production quantity objective a Production Performance Synthetic Indicator
(PPSI) was developed. The objective of the tool was to continually provide control
room engineers with an indication of production performance regarding quantity and
cost. To facilitate information sharing among the different layers of the enterprise, the
indicator is presented in different forms according to the different plant departments
(e.g. administration, management, maintenance, production, process). This helps all
the plant people have a common view of the production process.
Regarding quality, the main problem for the manufacturing of clinker is the control of
some parameters that affect the production process and hence clinker quality. These
parameters cannot be obtained in real-time as they can only be known after laboratory
tests. In PIKMAC a Quality Deviation Early Detector (QDED) was implemented to
predict clinker quality at one stage of the process taking into account parameters that
could be collected in a continuous way from previous stages of the process. QDED
helps improve process quality as continuous estimations of present clinker quality can
be made. Further, based on the data known at a given moment in a stage of the
process, the quality of future clinker production can be estimated provided that the
remainder parameters of the process remain unchanged. The indicator can also be used
to tune parameters of the different production stages given known parameters of
previous stages (the composition of the initial raw material for instance).
Figure 86: PIKMAC user interface
For the maintenance objective, the aim was to provide assistance to operators in a
plant that runs almost unattended during the night and weekends. This is called one220
man shift in the plant jargon. For this purpose, another module the Alarm Manager
Operator Assistant (AMOA) was developed. AMOA assesses the operator in order to
decide whether a given failure should be fixed immediately or not. This decision is not
easily taken as the economic optimum of the plant depends on several complex
constraints; electricity cost profile for the hours and day to come, cement storage,
production plan and availability of spare parts for repairing.
In a system like this, multiple different systems must be integrated to gather and to
deliver the information needed for module reasoning in a timely manner. The
framework selected for this purpose was that of ICa. Among others ICa was used to
integrate the following systems (apart from PIKMAC developed systems).
•
•
•
•
Alarms generated by the PROTOP digital system.
The industrial CIM21 database that recorded process parameters.
The automatic laboratory analysis.
The Incident Report Database where the problems of the plant were stored.
Figure 86 shows the user interface of PIKMAC.
221
Chapter 8
Conclusions and Future Work
223
8.1 Conclusions
This section presents the conclusions of the work carried out for this Thesis. The
results are grouped in different subsections according to their nature.
8.1.1
Specific Thesis Objectives
The introductory chapter of this document presented two Thesis objectives which are
reviewed in the following items.
•
•
The first one was that of co-developing a real-time CORBA ORB for its
adoption in distributed control systems. This objective has been completely
fulfilled as ICa ORB is a piece of software that has proven its usefulness in a
multitude of distributed control systems from different domains of control.
ICa ORB has been employed in large control systems and in embedded
systems. Also, it has been employed with success in systems directed by the
occurrence of events as well as in systems based on the time triggered
paradigm. That way, ICa demonstrates that it is capable of embracing a wide
spectrum of control systems which was a fundamental aim of this Thesis; the
demonstration that a single technology can be the basic infrastructure of many
different types of control systems, solving at the same time the problem of
integration suffered by this type of systems. Solving the integration problem is
not in itself a simple business and, in the industry, this problem alone can be
defined as harsh at a minimum.
The second objective was that of introducing a methodology which could ease
the work of implementing CORBA Control Systems. The approach of the
ICm methodology has been made from the point of view of domain
engineering. This has yielded several advantages: a) ICa can be employed as
the core technology to build single CORBA Control Systems by the use of
application engineering, b) the ICm methodology can be employed with a
multiple system scope point of view, being then possible to implement
frameworks on top of ICa ORB for the development of families of distributed
control systems for a control domain. Which alternative is used depends on
the development team and its objectives.
225
The previous items show that the Thesis objectives have been fulfilled. However,
many other important initially secondary objectives which are worth being mentioned
have also been fulfilled. The following sub-sections present these objectives.
8.1.2
Awareness of Standardization Bodies
The demonstration that the ICa technology can solve many problems of distributed
control systems is of great value per se but if these findings are done in isolation there
is no way to influence the industry to make use of it. For this reason, one of the
concerns during these last years has been to expose the ideas behind ICa in as many
forums as possible.
The main result of this effort of publicly exposing the ideas behind ICa can be found
in the Object Management Group. A great amount of work has been employed in
injecting the ideas of CORBA Control Systems into the people of OMG. In
consequence, sufficient interest has been raised to create a working group inside the
OMG with the objective of fostering the availability and suitability of specifications
for distributed control systems based on CORBA technology. The group’s name is
Control Systems Working Group and belongs to the Real-Time, Embedded and
Specialized Systems subgroup of the Platform Technology Committee of the OMG.
Figure 87 shows the web page for the working group at the OMG.
Figure 87: Control Systems WG
226web page at OMG's site
The creation of the working group took place during the HRTC project which
included technology dissemination as a primary objective. All the partners, who
include UPM (Spain), the Technical University of Vienna (Austria), the University of
Lund (Sweden) and SCILabs Ingenieros (Spain), made great efforts to contribute to
the dissemination and exploitation activities work package which the author of this
Thesis managed along with the hard real-time ORB work package.
It is important to embed future developments regarding CORBA Control Systems in a
standardization body. This Thesis work has always fostered an approach based in a
common interoperable technology which is vendor independent. That is why it is
important to let other people collaborate and promote further specifications.
8.1.3
Domain
Systems
Engineering
of
Substation
Automation
The SAS framework has been presented in the previous chapter and examples of its
development were presented in Chapter 6 while the ICa methodology was reviewed.
Once more, this domain engineering process to build substation automation systems
on top of ICa has been done with an integrative approach. The knowledge of the
domain was collected from the expertise of industrial partners in the domain and from
the IEC-61850 standard for substation automation. Gathering the knowledge only
from industrial partners could result in a valuable but otherwise particular vision of the
SAS domain. Instead of that, the idea was to build the framework around the IEC61850 standard and, at the same time, to take advantage of the deep knowledge of the
domain from industrial partners. This resulted in two main accomplishments:
•
•
A framework based on the IEC-61850 on top of ICa technology was built.
A contribution to the IEC-61850 was achieved as the mapping was introduced
to Technical Committee 57 of IEC-61850.
It has to be said that the framework was the first working implementation of the
standard on CORBA technology in the world.
8.1.4
Results for Third Parties
Comprehensibly, over these years, we have gained an understanding of CORBA
technology that goes beyond of what could be expected from professionals who
simply use the technology. Overall, this expertise is very valuable when dealing either
with large complex systems or with systems that try to push the technology close to its
227
limits (e.g. hard real-time CORBA systems). That expertise has been recognized by
companies that operate in different industrial environments which have requested the
assessment of the author for different projects. Some examples follow 13 .
•
•
•
Consultancy has been provided to a Spanish engineering company that
operates in the military area for the re-implementation of a N.A.T.O.
distributed hard real-time system.
One of the largest engineering firms in Europe requested assessment on the
available interoperability technologies for the implementation of the
forthcoming systems of the Single Sky for Europe project. This project is
related to air traffic management from Europe’s point of view and not from
the current single country perspective. It is a long term effort within a
development time frame of more than ten years.
One of the world’s biggest engineering companies from Germany requested
the implementation of an ORB over their automation platforms for the control
of product lines.
The existence of this market from real-world companies should be considered as very
important. It means that problems do exist; it means that the technology is capable of
solving them (at least of making it easier to deal with them) and, it also means that
there already is a real practical application of the ideas behind this Thesis. Thus, in
this work a contribution of added value to the European Union policies regarding
standardization, interoperability and coherence at the global level and, of improvement
of living conditions of Europeans by means of ameliorating the development of DCSs
has been achieved. Further, it has already contributed to the economic development
and growth of European companies as the items in the previous list demonstrate.
8.1.5
Research Support
Another result of this work is the long term collaboration between UPM and SCILabs
Ingenieros that is manifest in the signature of an open agreement for further
collaboration in the development of the ICa broker and its use in control system
research projects.
13
Company names are omitted for confidentiality matters.
228
8.2 Future Work
The existing work opens a range of possibilities for CORBA Control Systems that can
be addressed by further development efforts. Some of the future work has a general
cross-domain scope, as that of ICa, whereas other proposals of further work should be
carried out in the frame of concrete control domains.
8.2.1
CORBA Control System Services
The idea of CORBA Control System Services is that of extending the ICa architecture
by means of pre-built services that are useful for many control domains. ICa is only
the initial framework upon which many other useful services can be built. Services in
CORBA Control Systems would primarily be ICa objects or assets. A distributed
control system would be constructed by composition and configuration of services.
The reader should remember that ICa is important because it defines a precise way of
building distributed control systems. In the same way, CORBA Control Services
would aid to the development of DCSs by providing standard pieces to build upon.
This results in a more predictable way to build systems and leads to a simpler
definition of corporate standards for system development.
The services to be built can be further refined depending on the layer of control where
they perform their duty. For instance, at the control layer it is possible to think of
services for batch control processes, continuous control processes or discrete control
processes. Upper layers can be developed and deployed by means of manufacturing
services devised for production scheduling and supervision and even the topmost
layers can be implemented by the use of business services envisaged for the complete
plant operation. As an example, some possible services follow:
•
Data servers. Many DCSs today are built upon an architecture that relies on
data servers for system state distribution. This can be seen for instance in
Figure 14 of page 38 where the AVE power substation control system is
depicted. The control network interfaces with upper control layers by means
of an OPC server that collects and serves process data. The data server is an
interface with all PC technology that houses systems like HMI, gateways to
other protocols or even PC-based control systems. Although in CORBA
Control Systems there is no need of such interface, this type of architecture,
based on the data server, decouples the control layer from the rest. A data
server could be built on top of CORBA to perform such function. Also, many
vendor solutions interoperate with OPC. The ICa data server can also serve as
a gateway to this type of systems. For instance, in the traversal approach
229
•
•
•
•
•
•
proposed earlier in this document, the event service of CORBA could be used
as the basis for the specification of the ICa data server for distributed control
systems. The OMG has already done some OPC-aligned work in this area
[OMG 02c], [OMG 03f].
Alarm and event management services. Alarm and event management
services can be easily built either on top of ICa ORB or on top of ICa data
servers. Alarms or events are simple data that satisfy user-defined alarm
conditions. So it is possible to define what is an alarm or an event and what is
not. For instance, an alarm of some type is defined when variable A value is X
and / or variable B value is Y. The conditions can be made arbitrarily
complex. Alarm services must be complemented with those related to logging
for system execution tracing purposes.
History services. History services are a form of database services where
system information is stored for later retrieval. History services are important
because they help to make analysis on the plant data. For instance, trend
analysis in a power substation can be used to predict future power
consumption.
Diagnostic services. In very much the same way alarms can be defined, it is
possible to define invariants for the diagnosis of system elements. A system
can be designed in a way that some pressure measure must be between two
given values. The ICa diagnostic service supervises that such values are never
exceeded. Maintenance or other kind of alarms could be triggered in the alarm
service if the diagnostic service detects faulty conditions.
Reporting and analysis services. Information analysis is vital for the
production efficiency improvement of the plant. Thus, analysis and reporting
services are important as they allow tracking what is happening in the plant.
By data analysis, it is possible to determine if there is a chance of
improvement and to perform modifications in the plant.
Device integration Services. These are also called ICa Probes. Devices are
tweaked by manufacturers to interface to protocols of their choice. By
developing ICa Probes, it is possible to integrate every device into the ICa
network. An ICa probe acts as a gateway, showing on one side a standard ICa
interface defined in IDL and on the other the interface available for the device.
The mission of the probe is to act as a protocol translator.
HMI Services. Every plant is different. Today’s DCSs development suites
offer utilities for the design of HMIs in which custom screens for the DCSs
can be developed. In general, most of these utilities are built around OPC so a
data server gateway could integrate ICa with these tools. However, it is
possible to develop a protocol-independent suite for HMI development having
230
IIOP as its primary protocol but open to other protocol plug-ins. It would be a
HMI suite built on top of ICa ORB and its open connection architecture.
8.2.2
Development of Domain Specific Control Frameworks
In very much the same way a control framework for substation automation systems
was developed, it is possible to address other domains of distributed control systems.
By the use of the ICa methodology such frameworks can be reasonably built.
This section is intentionally presented after the section dedicated to CORBA Control
System Services. This is done so because CORBA Control Services have a crossdomain perspective. This is defined to mean that those services are useful for the
implementation of specific distributed control frameworks on top of ICa.
As already explained in the ICa methodology chapter, deep expert knowledge is
needed for the development of such frameworks inside an effort of domain
engineering. However, once the subsystems, assets and objects for a domain are
achieved, much can be extracted from the objects that model the domain. The author
believes there are opportunities in the Spanish and European markets for the
development of such frameworks. Three of such opportunities have already been
mentioned. In the case of the high speed railway, control technology is bought from
foreign companies. In the case of the Single Sky for Europe project, new systems with
a need for a very long lifespan and for change and evolution must be developed in the
years to come. Finally, in the case of automation, enhancements to the automatic
behaviour of networks of PLCs in production lines (e.g. hot-swapping of a PLC) can
be achieved with ICa technology.
8.2.3
The ICa IDE
The Integrated Control Environment (ICe) is a tool for the construction of ICa-based
systems. The current tools provided by ICa are command-line based. The visual
feedback and the interaction level provided by such tools are not very high. In order to
correct this situation and to improve development efficiency of ICa systems a
development environment can be built around ICa. To put an example, such an
environment will be something similar to the graphical interface tools that are
available in open-source license form for the GCC compiler or the GDB debugger. A
step forward these simple GUIs could be made by incorporating rapid application
development (RAD) techniques to this IDE such as the inclusion of wizards for the
231
generation of IDL, stubs and skeleton files of predefined systems. This alternative is
being explored by our research group in the context of the Eclipse environment and
the MDD (Model Driven Development) model of development. This work is starting
in the Aslab research group using the Eclipse platform as a framework.
8.2.4
Extending and Maintaining ICa ORB
The ICa ORB is the core of the CORBA Control Systems vision. Its maintenance and
evolution if crucial to the reminder systems as it is the nucleus of everything. In
consequence, a batch of activities should be spawned pursuant to these objectives.
Some of them are excerpted in the following items.
•
•
•
•
Dynamic Scheduling. ICa ORB addresses fixed time scheduling. Dynamic
scheduling poses much more complexity to system development as interaction
with an on-line scheduler is needed. The implementation of such a scheduler
for real-world systems is not a wise choice for most systems. However, it is
worth experimenting with this approach in the following years without ever
missing the current ability of ICa for fixed priority systems.
Transport plug-in developments. The development of ICa ORB plug-ins for
industrial protocols is by far a more pragmatic approach to future
developments than the inclusion of dynamic scheduling. At this time, systems
in the world are built around many different protocols which coexist the better
they can. Chapter 2 already showed examples in which many different
protocols are used in the same systems. Protocol plug-ins for ICa ORB can
take a twofold role. First they can be used to provide integration with systems
external to ICa. Secondly, they can be used in such a way that ICa ORB is
employed as the interoperability protocol on top of all the other protocols. The
second option leads to a pervasive use of ICa ORB as it makes developer
interoperability problems disappear. It is the first stage towards the
generalized use of CORBA technology for the implementation of distributed
control systems.
Porting. This a maintenance and evolution work needed periodically. New
versions of operating systems are released periodically and porting to new
operating systems is ordinary work in software development. To this future
work, great benefit can be taken from the ICa ORB architecture in which a
specific portability layer has been designed to isolate operating services from
the core of ICa ORB functionality. This means porting ICa is a matter of
extending the scope of the portability layer to new systems.
Asynchronous Method Invocations. The Messaging specification of
CORBA introduced a communications model which allows asynchronous
232
•
8.2.5
invocation of requests for the static interface of CORBA. This permits the
invocation of two-way requests in an asynchronous way via callback or poller
objects. This feature can be interesting for some ICa applications.
Automatic Management of Periodic, Sporadic and Aperiodic Tasks. This
are extensions to be included in ICa in the context of the proposal presented in
Section 5.13 and of the project COMPARE.
ICa Pattern Language
Although some effort has been done in capturing the patterns underlying the ICa
applications it is necessary to continue this work and elaborate the ICa Pattern
Language (ICpl) based on the pattern schema described in [Sanz 03a].
233
GLOSSARY
Abstract Class
A specialized class used solely for subtyping. It defines a
common set of behaviors to be inherited by its subtypes. It
has no instances. (Synonymous with Virtual Class in C++)
Abstract Data Type
A data type defined to model the data characteristics of realworld objects. An ADT provides a public interface via its
permitted operations, but the internal representation and
implementation of this interface are private.
Abstraction
The act of concentrating the essential or general qualities of
an object or objects. The resulting concept embodies the
"essence" of the objects under consideration.
Acknowledged Data
Transfer
The transmission of data from a source endpoint to one
endpoint, or, in the case of multicast, more than one
endpoint; and the subsequent response indicating the status of
the data transmission.
Activation
Copying the persistent form of methods and stored data into
an executable address space to allow execution of the
methods on the stored data.
Active Agent
An object with a self-owned thread of control.
Actor
An external agent that interacts with an application or
system. This is also a model for concurrent programming. In
some context this is a synonym of Agent.
Address
Field, or fields, in a message identifying both the source and /
or destination of the message.
ADL
Architecture Description Language.
Agent Description Language.
235
ADT
Abstract Data Type
Agent Description
Language
A language to provide specification mechanisms for agent
implementation.
Analysis
The process of developing a specification of what a system
does and how it interacts with its environment.
API
Application Program Interface
Application Facilities
Common facilities that are useful within a specific
application domain.
Application Objects
Applications and their components that are managed within
an object-oriented system.
Application
Programming Interface
The programming interface used to access and control a
library or program.
Application
A program or a set of programs that provides functionality to
the end user. Also refers to specific Algorithm(s)
implemented in an IED.
Architectural Domain
A domain for architectures, i.e. an area of knowledge defined
by a family of related architectures.
Architectural Style
A common core design shared by a collection of software
architectures.
Architecture Description Graphic and textual languages, standards, and conventions
Language (ADL)
used to represent a software architecture. ADLs are usually
related to DSSAs. ADLs are intended to be used for building
models during the development of new systems based on
existing architectures or architectures in an Architectural
Domain.
Architecture
A high-level description of the organization of functional
responsibilities within a system. Many different levels of
architectures are involved in developing software systems,
from physical hardware architecture through the logical
architecture of an application framework.
236
Association
1. An Association is established when a client-server “linkup” is made and is manifested in the communication path
established between a client and a server for the exchange of
messages. The Association is closed when the client-server
link is Concluded or Aborted. Object association.
2. Meaningful links between objects. A person associated
with a company creates the concept of employment.
Asynchronous Event
Event that occurs independently of the execution of an
application.
Asynchronous
Interaction
An interaction between schedulable units (processes and/or
threads) in which, after a schedulable unit invokes an
operation to take part in the interaction, control is allowed to
return to the schedulable unit before the completion of the
interaction.
Asynchronous Message Asynchronous message communication provides the
Communication
capability for objects to send messages, even without the
existence of the receiving object at the instant the message is
sent. The receiving object can retrieve messages at its
convenience. There is no blocking or synchronization
required between objects. Asynchronous message
communication is a foundation for constructing true
concurrent computing environments.
Asynchronous Request
A request in which the client object does not pause or wait
for delivery of the request to the recipient; nor does it wait
for the results.
Atomic Broadcast
The transfer of a broadcast message that is either guaranteed
to be received by all possible receivers or it is not received by
any.
Atomicity
The property that ensures an operation either changes the
state associated with all participating objects consistent with
the request, or changes none at all. If a set of operations is
atomic, then multiple requests for those operations are
serializable.
237
Attribute
An identifiable association between an object and a value. An
attribute A is made visible to clients as a pair of operations:
get A and set A. Read only attributes only generate a get
operation. A characteristic or property of an object. Usually
implemented as a simple data member or as an association
with another object or group of objects.
Bi-directional
transaction
A transaction in which a request and possibly data are
conveyed from a Client to a Server and in which a response
and possibly data are returned to the Client from the Server.
Binding
The selection of the method to perform a requested service
and of the data to be accessed by that method. (also Dynamic
Binding and Static Binding)
Broadcast
A message placed onto a Communication Network intended
to be read and acted on, as appropriate, by any device
connected to the network. A Broadcast message will
typically contain the sender’s address and a Global recipient
address. Example Time Synchronising.
CCS
CORBA Control Systems.
Complex Control Systems.
Class Hierarchy
Embodies the inheritance relationships between classes.
Class Inheritance
The construction of a class by incremental modification of
other classes.
Class Member
A method or an attribute of a class.
Class Method
A class method defines the behavior of the class. Such a
method performs tasks that cannot or should not be done at
the instance level, such as providing access to class attributes
or tracking class usage metrics.
Class Object
An object that serves as a class. A class object serves as a
factory. (Factory)
Class
A pattern that can be instantiated to create multiple objects
with the same behavior. An object is an instance of a class.
238
Types classify objects according to a common interface;
classes classify objects according to a common
implementation.
Classification
The act of determining which class or type applies to a
specific object.
Client
An object that requests a service from a server object in a
client/server relationship. The code or process that invokes
an operation on an object.
Client/server
A relationship between a client that requests services and
servers that provide services. This relationship is paralleled
in an object oriented environment by message senders and
receivers.
CNI
Communications Network Interface
Collaboration
1. Two or more objects that participate in a client/server
relationship in order to provide a service.
2. Collaboration is concerned with the interactions between
agents in a multiagent system when the whole system is also
considered as an agent with certain structure of system's
global state. Particularly, it is concerned with the
relationships between individual agents' mental structures
and internal states and the system's collective mental
structure and state. For example, a collaborative model of
multiagent systems may contain a model of system's global
intention and individual agent's intention, and we can talk
about congruence (that is the consistency between an agent's
behaviour and the whole system's global goal or intention)
and coherence (that is the consistency between an agent's
internal state, such as intention, and the system's goal or
intention).
COM
Common Object Model
Common Facilities
Facilities useful in many application domains and which are
made available through Object Management Architectures
(OMA)-compliant class interfaces. (Application Facilities)
239
Common Object Model
COM. Microsoft standard for object management
Common Object
Request Broker
Architecture
CORBA. A specification for objects to locate and activate
one another through an object request broker. The CORBA
specification facilitates object request brokers from different
vendors to interoperate.
Communication
Controller
A Communication Controller is a virtual device within an
IED that provides communication services related to clientserver associations.
Communications Stack
Hierarchy of communications software. For example the 7
Layer stack of the ISO Reference Model where each layer
performs a specific functional role in Open Systems
Interconnection communication.
Component
A conceptual implementation notion. A component is an
object that is considered to be part of some containing object.
Classes, systems or subsystems that can be designed as
reusable pieces. These pieces can then be assembled to create
various new applications. Sometimes used to refer to
software modules with plug and play behavior.
Composition
The creation of an object that is an aggregation of one or
more objects.
Compound Object
A conceptual notion. A compound object is an object that is
viewed as standing for a set of related objects.
Concrete Class
A class or type that can have instances. (Contrast with
Abstract Class).
Conformance Test
This test verifies that an object communication interface(s)
complies with the specified requirements.
Conformance
A relation defined over types such that type x conforms to
type y if any value that satisfies type x also satisfies type y.
Connection
An association established between functional units for
conveying information.
240
Connectionless
The transport of a single datagram or packet of information
from one network node to a destination node, or multiple
destination nodes, without the establishment of a network
connection.
Connector
A n-ary relation defined over components. Usually employed
in relation with software architecture modeling and Æ
ADLs.
Constraint
A relational or behavioral restriction or limit. Usually
regarded as a property that must always hold true.
Constructor
A method that is called when a new instance is created.
Constructor methods are used to initialize the new instance.
Container Class
A class designed to hold and manipulate a collection of
objects.
Context-Independent
Operation
An operation in which all requests that identify the operation
have the same behavior. (In contrast, the effect of a contextdependent operation might depend upon the identity or
location of the client object issuing the request.)
Contract
Defines the services provided by a server, along with the preconditions and post-conditions that apply to the use of those
services.
CORBA
Common Object Request Broker Architecture
Coupling
A dependency between two or more classes, usually resulting
from collaboration between the classes to provide a service.
Loose coupling is based on generic behavior and allows
many different classes to be coupled in the same way. Tight
coupling is based on more specific implementation details of
the participating classes and is not as flexible as loose
coupling.
CRC
Cyclic Redundancy Check. A CRC is performed for each
frame and the value is included in that frame when it is
transmitted. The CRC check calculation may be simple or
complex depending on the protocol being used. The CRC
241
value is used by the recipient communication interface to
check and if possible correct errors incurred during
transmission of that frame.
CSMA/CD
Carrier Sense Multiple Access/Collision Detection
Data Item
A single piece of information to be communicated.
Data Type
A categorization of values, operations and arguments,
typically covering both behavior and representation (e.g., the
traditional non-OO programming language notion of type).
Datagram
A datagram contains within the message all the information
for transmission without the requirement for establishing a
network connection.
DCE
Distributed Computing Environment
DCOM
Distributed Component Object Model Æ Component Object
Model
Deferred Synchronous
Request
A request where the client does not wait for completion of
the request, but does intend to accept results later. Contrast
with synchronous request and one-way request.
Delegation
The ability of a method to issue a request in such a way that
self-reference in the method performing the request returns
the same object(s) as self-reference in the method issuing the
request.(ÆSelf-Reference)
The ability of an object to issue a request to another object in
response to a request. The first object therefore delegates the
responsibility to the second object.
Derivation
The act of subclassing an existing class to define a new
subclass. (Æ Inheritance)
Derived Class
The class created through inheritance. A derived class
inherits the methods and attributes of its superclass(es) and
usually adds its own to distinguish its capabilities or services.
242
Design Pattern
A pattern that specifies the way to construct something to
satisfy some requirements (equilibrating forces) in some
context.
Design
A process that uses the products of analysis to produce a
specification for implementing a system. Also the result of
this process. Æ Design Pattern Æ Pattern
Destructor
A method involved whenever an object is ready to be
destroyed. It is usually implemented to revise the actions that
were performed during initialization, such as recovery of
allocated resources.
Device
A mechanism or piece of equipment designed to serve a
purpose or perform a function.
Distributed Computing OSF software specification and implementation to support
Environment (DCE)
development of distributed applications. (Æ Open Software
Foundation, Æ Remote Procedure Call)
Distributed Object
Computing (DOC)
A computing paradigm that distributes cooperating objects
across a - possibly heterogeneous - network and allows the
objects to interoperate as a unified whole.
DLL
Æ Dynamic Link Library.
DOC
Æ Distributed Object Computing.
Domain
A formal boundary that defines a particular subject or area of
interest. The HRCT domain is the domain of software
intensive, distributed, complex process control.
Domain Expert
A person who has special skill or knowledge of a particular
domain.
Domain Model
A model (terminology and semantics) that characterize the
elements, processes, and relationships within a family of
related systems.
Domain-specific
software architecture
An architecture that captures architectural commonality of
multiple, related systems, i.e., systems within the same
243
(DSSA)
domain.
DRE
Distributed Real-time and Embedded
DSOM
Distributed System Object Model. Æ System Object Model.
DSSA
Domain-Specific Software Architecture.
Dynamic Binding
Binding that is performed after a request is issued. (Æ
Binding)
Dynamic Classification
Classification of an object at runtime. This implies that an
object's classification can change over time.
Dynamic Invocation
Constructing and issuing a request whose signature is not
known until runtime.
Dynamic Link Library
A dynamically loaded run-time library.
Dynamic Object-Based The end-user functionality provided by one or more
Application
programs consisting of interoperating objects.
EAI
Enterprise Application Integration
Embedding
Creating an object out of a non-object entity by wrapping it
in an appropriate shell. (Æ Wrapper)
Encapsulation
The technique used to hide the implementation details of an
object. The services provided by an object are defined and
accessible as stated in the object contract. (Often used
interchangeably with Information Hiding)
Enterprise Modeling
A technique for modeling an entire business enterprise from
the business manager's point of view. An enterprise model is
composed of the objects, events and business rules that
describe the enterprise. Separate but related business systems
can be built from this model to enhance the efficiency and
consistency of the operation of the enterprise.
Environment
The totality of circumstances surrounding an agent or group
of agents, especially we can consider the physical and social
244
environment. Physical Environment: The combination of
external physical conditions that affect and influence the
growth, development, actions and survival of agents. Social
Environment: The complex of social and cultural conditions
affecting the nature of an agent or a community.
ERP
ERP (enterprise resource planning) is an industry term for the
broad set of activities supported by multi-module application
software that help a manufacturer or other business manage
the important parts of its business, including product
planning, parts purchasing, maintaining inventories,
interacting with suppliers, providing customer service, and
tracking orders. ERP can also include application modules
for the finance and human resources aspects of a business.
Typically, an ERP system uses or is integrated with a
relational database system.
Event
A significant change in the environment or the state of an
object that is of interest to another object or system.
Externalized Object
Reference
An object reference expressed as an ORB-specific string.
Suitable for storage in files or other external media.
Factoring
The process of extracting the common properties or behavior
from a group of objects so that the common elements can be
propagated to a common subclass. Factoring eliminates
duplication.
Factory
A concept that provides a service for creating new objects.
Fault-Tolerance
The characteristic of a system that allows it to handle the loss
of a particular component without interrupting normal
operations.
Field Bus
A communications network shared by multiple,
communicating, physical nodes. Typical technology in lower
layers of process control systems.
FMS
Fieldbus Messaging Specification.
Formal Parameter
A named local object used as an argument to an operation.
245
The value of the object (actual parameter) is assigned by the
client who runs the method.
Frame Format
A template for the actual message to be transmitted. It
typically defines the Header, Start of frame, Destination
address, Source address, length/type of contained data, the
actual data, padding bytes if required and some form of CRC
data, then end of frame marker.
Framework
A set of collaborating abstract and concrete classes that may
be used as a template to solve a specific domain problem.
Function
Functions are tasks that are performed in the application by
the components of the CCS.
Functional
Decomposition
The process of refining a problem solution by repeatedly
decomposing a problem into smaller and smaller steps. The
resulting steps are then programmed as separate modules.
Functional Interface
Interfaces that define the operations invoked by users of an
object service. The audience for these interfaces is the service
consumer, the user of the service. These interfaces present
the functionality (the useful operations) of the service. An
Object Service Definition.
Gateway
A network interconnection device which supports the full
Stack of the relevant Protocol and can convert to a non 7
Layer Protocol for asynchronous transmission over Wide
Area Networks.
Generalization
The inverse of the specialization relation.
Generic Object
An object (relative to some given Object Service) whose
primary purpose for existence is unrelated to the Object
Service whose interface it carries. The notion is that the
Object Service is provided by having (in principle) any type
of object inherit that object service interface and provide an
implementation of that interface. An Object Service Domain.
Generic Operation
The concept that an operation is generic if it can be bound to
more than one method.
246
Graphical User Interface Any interface that communicates with the user, primarily
through graphical icons.
GUI
Graphical User Interface.
Handle
A value that identifies an object.
HMI
Human Machine Interface.
Hub
A Hub is a Communications Network component providing
multiple ports, each interfacing to a separate media link in a
star topology.
ICa
Integrated Control Architecture
ICa ORB
The Integrated Control Architecture real-time CORBA
broker.
ICe
Integrated Control Environment
ICm
Integrated Control Methodology
ICpl
ICa Pattern Language
IDL
Interface Definition Language.
IED
Intelligent Electronic Device
Implementation
Definition Language
A notation for describing implementations. The
implementation definition language is currently beyond the
scope of the ORB standard. It may contain vendor-specific
and adapter-specific notations.
Implementation
Inheritance
The construction of an implementation by incremental
modification of other implementations. The ORB does not
provide implementation inheritance. Implementation
inheritance may be provided by higher level tools.
Implementation Object
An object that serves as an implementation definition.
Implementation objects reside in an implementation
247
repository.
Implementation
Repository
A storage place for object implementation information.
Implementation
1. The development phase in which the hardware and
software of a system become operational.
2. A definition that provides the information needed to create
an object and allow the object to participate in providing an
appropriate set of services. An implementation typically
includes a description of the data structure used to represent
the core state associated with an object, as well as definitions
of the methods that access that data structure. It will also
typically include information about the intended interface of
the object.
Implicit Invocation
A mechanism of invocation in which the invoker does not
know anything about the invokee. Usually implemented
through a callback registration mechanism in the invoker or a
broadcast of an event to all possible invokees.
Inheritance
The construction of a definition by incremental modification
of other definitions. (Æ Implementation Inheritance)
Initialization
Setting the initial attribute values of a new object.
In-Line Method
A mechanism that allows the compiler to replace calls to the
method with an expansion of the method code.
Instance Name
An identifier associated with and designating an instance.
Instance Variable
A variable that contains a value specific to an object instance.
Instance
An object created by instantiating a class. An object is an
instance of a class. A functional unit comprising an
individual named entity having the attributes of a defined
class and providing defined services.
Instantiation
Object creation. The creation of an instance of a specified
class.
248
Integrability
The capability of a system of being used as part of another,
bigger CCS system.
Interface Definition
Language
When used in conjunction with an ORB, IDL statements
describe the properties and operations of an object. IDL is
used to specify the public interface of a CORBA object.
Intelligent Electronic
Device
An IED is any device incorporating one or more processors,
with the capability to receive, process or send, data / control
from, or to, an external source.
Interface Inheritance
The construction of an interface by incremental modification
of other interfaces. The CORBA IDL provides interface
inheritance.
Interface Type
A type that is satisfied by any object (literally, by any value
that identifies an object) that satisfies a particular interface.
(Æ Object Type)
Interface
1. A shared boundary between two functional units, defined
by functional characteristics e.g.- common physical
interconnection characteristics, signal characteristics or other
characteristics as appropriate, and the provision of a declared
collection of services.
2. A description of a set of possible uses of an object.
Specifically, an interface describes a set of potential requests
in which an object can meaningfully participate. (Æ also
Object Interface, Principal Interface and Type Interface)
Interoperability
1. The ability of two systems to exchange services.
2. The ability for two or more ORBs to cooperate to deliver
requests to the proper object. Interoperating ORBs appear to
a client to be a single ORB.
Invariant Relation
A relation that cannot be changed so long as it has instances.
IP
Internet Protocol. The TCP/IP standard protocol. IP defines
the datagram that provides the basis of connectionless packet
delivery. It includes control and error message protocol
providing the equivalent functions to Network services,
249
Layer 3, of the OSI Reference Model.
ISO
International Organization for Standardization
LAN
Local Area Network
Language Binding or
Mapping
The means and conventions by which a programmer writing
in a specific programming language accesses ORB
capabilities.
Legacy System
A previously existing system or application.
Leveling
The process of grouping information or concepts at various
levels of increasing detail. The top-most level is general in
nature and each successive level adds more detail until all
aspects of the given subject matter have been explained in
detail.
Life-Cycle Service
The Object Life-Cycle Service provides operations for
managing object creation, deletion, copying and equivalence.
An Object Service Definition.
Link
1. Connection between two processing entities.
2. Relation between two objects (a concept).
Local Area Network
A communications network which typically covers the area
within a building or small industrial complex.
Log
A record, a journal, of chronologically ordered data e.g.
Events + Time Tags + Annotations.
Managed Object
Clients of System Management services, including the
installation and activation service and the operational control
service (dynamic behavior). These clients may be application
objects, common facilities objects, or other object services.
The term is used for compatibility with system management
standards (the X/Open GDMO specification and ISO/IEC
10164 System Management Function, Parts 1 to 4). An
Object Service Definition.
Management
(MIS) An organized assembly of resources and procedures
250
Information System
required to collect, process and distribute data for use in
decision making. A computer system for a business or other
organization which collects and analyzes data from all
departments, and is designed to provide an organization's
management with up-to-date information (such as financial
reports, inventory, etc.) at any time.
Mapping
1. A set of values having defined correspondence with the
quantities, or values, of another set.
2. A rule or process, the OO equivalent of a mathematical
function. Given an object of one set, a mapping applies its
associative rules to return another set of objects. Member
Function (Æ Method)
MES
Manufacturing execution systems. Systems that use network
computing to automate production control and process. By
downloading "recipes" and work schedules and uploading
production results, MESs bridge the gap between business
and plant-floor or process-control systems.
Message
The mechanism by which objects communicate. A message
is sent by a client object to request the service provided by
the server object.
Meta-Model
A model that defines other models.
Meta-Object
An object that represents a type, operation, class, method or
object model entity that describes objects.
Meta-Type
A type whose instances are also types.
Method
1. In systems development, a cohesive set of rules, methods
and principles used to guide the modeling and development
of software systems.
2. Code that can be executed to perform a requested service.
Methods associated with an object are structured into one or
more programs.
Method Resolution
The selection of the method to perform a requested operation.
MIDL
Microsoft Interface Definition Language. IDL used by
251
Microsoft Windows applications (Æ
Language)
Interface Definition
MMS
Manufacturing Message Specification - [ISO 9506]
Multi-cast
A message placed onto the communication network intended
for a limited set of recipients. A Multi-cast message will
typically contain the sender’s address and an address field
defining a limited set of recipient’s addresses.
Multiple Classification
Ability of an object to belong to more than one type.
Multiple Inheritance
of more than one other definition.
Object
A combination of a state and a set of methods that explicitly
embodies an abstraction characterized by the behavior or
relevant requests. An object is an instance of a class. An
object models a real world entity and is implemented as a
computational entity that encapsulates state and operations
(internally implemented as data and methods) and responds
to requests for services. An object is a self-contained
software package consisting of its own private information
(data), its own private procedures (private methods), which
manipulate the object's private data, and a public interface
(public methods) for communicating with other objects.
Object Adapter
The ORB component that provides object reference,
activation and state-related services to an object
implementation. There may be different adapters provided
for different implementations.
Object Attribute
A Field, or, a category or value of data that, together with
other attributes, specify the services or data values related to
the function and performance of an Object.
Object Broker
Object Creation
An event that causes an object to exist that is distinct from
any other object.
252
Object Interface
A description of a set of possible uses of an object.
Specifically, an interface describes a set of potential requests
in which an object can meaningfully participate as a
parameter. It is the union of the object's type interfaces.
Object
Library/Repository
A central repository established expressly to support the
identification and reuse of software components, especially
classes and other software components.
Object Management
Architecture
The generic architectural design proposed by the OMG. The
CORBA specification is the first standard of technology for
OMA
Object Management
Group
A non-profit industry group dedicated to promoting objectoriented technology and the standardization of that
technology.
Object Modeling
Technique
An object-oriented systems development life cycle developed
by General Electric. Now being integrated by its developer
Rumbaugh into a new method named Unified Modeling
Language.
Object Reference
A value that precisely identifies an object. Object references
are never reused to identify another object.
Provides the means by which objects make and receive
requests and responses. The middleware of distributed object
computing that provides a means for objects to locate and
activate other objects on a network, regardless of the
processor or programming language used to develop and
implement those objects.
Object Services
The basic functions provided for object lifecycle
management and storage such as creation, deletion,
activation, passivation, identification and location.
Object State
The current information about an object that determines its
behavior.
Object Type
A type the extension of which is a set of objects (literally, a
set of values that identify objects). In other words, an object
253
type is satisfied only by (values that identify) objects. (Æ
Interface Type)
Object Wrapper
The result of encapsulating a set of services provided by a
non OO application or program interface in order to treat the
encapsulated application or interface as an object.
Object-Based
A programming language or tool that supports the object
concept of encapsulation, but not inheritance or
polymorphism.
Object-Oriented
Analysis
The process of specifying what a system does by identifying
domain objects and defining the behavior and relationships of
those objects.
Object-Oriented
Business Engineering
A framework and discipline used to effectively model
business processes. It involves identifying business objects,
processes, structures, rules, policies, organizational structure
and authority, location and logistics, technology and
applications. Its goal is to produce precise descriptive models
of business objects that can be converted into reusable and
easily modifiable software components.
Object-Oriented Design
The process of developing an implementation specification
that incorporates the use of classes and objects. It encourages
modeling the real world environment in terms of its entities
and their interactions.
Object-Oriented
Any language, tool or method that focuses on modeling real
world systems using the three pillars of objects
encapsulation, inheritance and polymorphism. Also OO.
OMA
Object Management Architecture
Oneway Request
A request in which the client does not wait for completion of
the request, nor does it intend to accept results. Contrast with
deferred synchronous request and synchronous request.
Ontology
An ontology is an explicit specification of the structure of a
certain domain (e.g. e-commerce, sport, ...). In practical
terms this includes a vocabulary (i.e. a list of logical
254
constants and predicate symbols) for referring to the subject
area, and a set of logical statements expressing the
constraints existing in the domain and restricting the
interpretation of the vocabulary. Ontologies therefore provide
a vocabulary for representing and communicating
knowledge.
Open Software
Foundation
Non-profit standarization organization dedicated to promote
open software standards, i.e. OSF/Motif, OSF/DCE and
OSF/1. Now has joined forces with X/Open to constitute the
Open Group.
Operation
A service that can be requested. An operation has an
associated signature, which may restrict which actual
parameters are possible in a meaningful request.
Operation Name
A name used in a request to identify an operation.
ORB
Æ Object Request Broker
ORB Core
The ORB component that moves a request from a client to
the appropriate adapter for the target object.
OS
Operating System
OSF
Open Software Foundation
OSF/1
UNIX-like operating system based on a microkernel
architecture. (Æ Open Software Foundation)
OT
Object Technology
Overloaded Operation
Multiple methods of the same name, each having a unique
signature. This allows the methods of the same name to be
invoked with various argument types.
Paradigm
A broad framework for thinking about and perceiving reality.
A theoretical, philosophical model composed of identifiable
theories, laws and generalizations used in defining and
solving problems.
255
Parallel Processing
The simultaneous execution or computation of two or more
programs or operations.
Parameter Passing Mode Describes the direction of information flow for an operation
parameter. The parameter passing modes are IN, OUT and
INOUT.
Parameters
Parameters are data that define the behaviour of functions.
Parameterized Class
A class that allows users to declare member functions and
data members of "Some Type," which can be used as a
template for declaring specialized subclasses that supply the
"Missing" types.
Partition
Decomposing a type into its disjoint subtypes.
Pattern
A pattern describes a problem, a solution to a problem, and
when to apply the solution. Patterns may be categorized in
several ways; by example as design patterns, business
process patterns and analysis patterns. Æ Design Pattern
Persistent Object
An object that can survive the process or thread that created
it. A persistent object exists until it is explicitly deleted.
Plug and Play
A type of component that needs little modification to be
integrated into a system.
Pluggable Transport
A CORBA transport layer that can be added or eliminated in
run-time.
Point to Point
A dedicated communication link between two nodes only.
Pointer
A variable that can hold a memory address of an object.
Polymorphic Operation
The same operation implemented differently by two or more
types.
Polymorphism
The concept that two or more types of objects can respond to
the same request in different ways.
Portable Object Adapter An object adapter is the primary means for an object
256
(POA)
implementation to access ORB services such as object
reference generation. An object adapter exports a public
interface to the object implementation, and a private interface
to the skeleton. It is built on a private ORB-dependent
interface. The Portable Object Adapter offers functionality
enough to build portable servers.
Post-Condition
A constraint that must hold true after the completion of an
operation.
Pre-Condition
A constraint that must hold true before an operation is
requested.
Presentation Layer
Layer 6 of the ISO Reference Model.
Principal Interface
The interface that describes all requests in which an object is
meaningful.
Private
A scoping mechanism used to restrict access to class
members so that other objects cannot them.
Property
An attribute, the value of which can be changed.
Protected
A scoping mechanism used to restrict access to class
members.
Protection
The ability to restrict the clients for which a requested service
will be performed.
Protocol
A set of rules governing the information transfer within a
communications network. Protocols perform Data Link
Control by defining frame format(s), timing, error correction
and Handshaking.
Public
A scoping mechanism used to make member access available
to other objects.
Query
An activity that involves selecting objects from implicitly or
explicitly identified collections based on a specific predicate.
Rapid Prototyping
The iterative process of quickly developing a prototype of an
257
application, usually with the aid of specific GUI-building
tools. This process is used to help uncover unknown details
of the system under consideration, and to build the system in
small increments.
Redundancy
Refers to spare or duplicate functionality that allows a
System to continue to operate without degradation of
performance in the event of single failure. e.g. a blown fuse.
Reference Architecture
An architectural description for a family of applications that
describes functional components, connections, protocols, and
control. A reference architecture generally consists of a
partially-specified system composed of generic or abstract Æ
Components, that are replaced by real components when the
architecture is instantiated for an actual system.
Referential Integrity
The property that ensures that a handle which exists in the
state associated with another object reliably identifies a
single object.
Relation
An object type that associates two or more object types. A
relation is how associations are formed between two or more
objects.
Remote Procedure Call A protocol which allows a program running on one host to
(RPC)
cause code to be executed on another host without the
programmer needing to explicitly code for this. RPC is an
easy and popular paradigm for implementing the clientserver model of distributed computing. An RPC is
implemented by sending request message to a remote system
(the server) to execute a designated procedure, using
arguments supplied, and a result message returned to the
caller (the client). There are many variations and subtleties in
various implementations, resulting in a variety of different
(incompatible) RPC protocols.
Repository
Usually a central location used to store and organize software
components and related definitions, rules, etc. (Æ Object
Library/Repository)
Request
An event consisting of an operation and zero or more actual
258
parameters. A client issues a request to cause a service to be
performed. Also associated with a request are the results that
can be returned to the client. A message can be used to
implement (carry) a request and/ or a result.
Requirements
A document describing what a software system does from a
user's point of view. This document is input into the objectoriented analysis process, where it will be transformed into a
much more precise description.
Responsibility
A service or group of services provided by an object; a
responsibility embodies one or more of the purposes of an
object.
Result
The information returned to the client, which can include
values as well as status information, indicating that
exceptional conditions were raised in attempting to perform
the requested service.
Reuse
Reuse is the process of locating, understanding and
incorporating existing knowledge, design and components
into a new system. Reuse should occur at all levels of system
development analysis, design, implementation, testing,
documentation and user training.
Role
A sequence of activities performed by an agent. A portion of
the social behavior of an agent that is characterized by some
specificity such as a goal, a set of attributes (for example
responsibilities, permissions, activities, and protocols) or
providing a functionality/service. Typically related to the
realization of a pattern or provision of a function.
RTOS
Real-time Operating System
RTU
Remote Terminal Unit- typically a station in a SCADA
system, an RTU acts as an interface between the
communication network and the plant equipment.
Scalability
The ability of a system to grow without sacrificing
performance. The Scalability is the criteria for a universal
and cost effective CCS, taking into account the varying
259
functionality, plant sizes and magnitude ranges.
Schema
A formal presentation with a defined set of symbols and rules
that govern the formation of a representation using the
symbols. There are many different kinds of schema,
including object, event and activity schemas.
Security Domain
A subset of computational resources used to define a security
policy.
Self-Reference
The ability for a method to identify the target object for
which it was invoked. This notion is referred to by the key
words "self " in Smalltalk and "this" in C++ or Java.
Semantics
The meaning - the essence - of the definition.
Server
The entity that provides a service that can be requested.
Server Class
A Server Class comprises of the external visible behaviour of
an Application process.
Server Object
An object providing response to a request for a service. A
given object might be a client for some requests and a server
for other requests. (Æ Client)
Service
A functional capability of a resource which can be modified
by a sequence of service primitives.
Service Cycle
The complete process -and the time elapsed on it- from the
issue of a request till the response to it.
Service Primitive
Abstract, implementation independent, representation of an
interaction between the service user and the service provider.
Service
A computation that can be performed in response to a
request.
Signature
Defines the parameters of a given operation including their
number order, data types and passing mode; the results, if
any; and the possible outcomes (normal vs. exceptional) that
might occur.
260
Single Inheritance
of one definition. (Æ Multiple Inheritance)
Skeleton
The object-interface-specific ORB component that assists an
object adapter in passing requests to particular methods.
Specialization
A class x is a specialization of a class y if x is defined to
directly or indirectly inherit from y.
State
The information about the history of previous requests
needed to determine the behavior of future requests.
State Machine
A formal description of the functionality, responses, actions
and re-actions, as a series of discrete, linked states, together
with the criteria governing the transition from one state to
another specific state.
State-Modifying
Request
A request that by performing the service alters the results of
future requests.
Static Binding
Binding that is performed prior to the actual issuing of a
request. (Æ Binding)
Static Invocation
Constructing a request at compile time.
Static Member Function In C++, a function declared part of a class declaration. These
functions can be invoked independent of any instances of the
class.
Strong Typing
A language characteristic that requires an explicit type
declaration for every value or expression. Strong typing
makes static binding feasible.
Stub
A local procedure corresponding to a single operation that
invokes that operation when called. Subclass (Æ Subtype)
Sub–functions
Sub–functions are sub parts of a main function. A sub–
function may be shared by more than one main function.
Subscribed data
Data that a Client has requested to be supplied on a regular
261
basis, or when trigger condition(s) are satisfied.
Subtype
A specialized or specific object type.
Superclass
A class that provides its methods and attributes to another
class derived from it via inheritance.
Switch
An active Communications Network device that facilitates
the exchange of data between two devices, on different LAN
segments, by dynamically connecting the two LAN segments
together only as and when data transfer is required.
Effectively multiplies the available bandwidth, allowing
LAN segments to run in parallel.
Synchronous Request
A request in which the client object pauses to wait for
completion of the request.
System
Within this document, “System” refers to CORBA Control
Systems, other types of system will be identified by their
prefix name.
System Object Model SOM is IBM’s CORBA-compliant object request broker for
(SOM)
a single address space architecture. A similar distributed
system object model framework exists to allow objects to
communicate across address spaces and networks. An
implementation of CORBA by IBM.
Target Object
An object that receives a request. (Synonymous with Server
Object)
TCP/IP
Transmission Control Protocol/Internet Protocol. A suite of
protocols which together provide the functionality up to layer
4, of the ISO OSI Reference Model, without exact layer for
layer correspondence.
Transient Object
An object whose existence is limited by the lifetime of the
process or thread that created it.
Type
A predicate (Boolean function) defined over values that can
be used in a signature to restrict a possible parameter or
characterize a possible result. Types classify objects
262
according to a common interface; classes classify objects
according to a common implementation.
Type Interface
Defines the requests in which instances of this type can
meaningfully participate as a parameter. Example: If given
document type and product type in which the interface to
document type comprises "edit" and "print," and the interface
to product type comprises "set price" and "check inventory,"
then the object interface of a particular document that is also
a product comprises all four requests.
UCA 2.0
Utility Communications Architecture version 2.0 describes
the concepts of standardised models for Power System
Objects.
UML
Unified Modeling Language.
Unified Modelling
Language (UML)
Standardised constructs and semantics for diagrams,
including state machines, which are used to describe / specify
the functionality of software-intensive artifacts and systems.
Unsolicited Data or
Unsolicited Message
A Message or Data which is supplied to a Client, or Clients,
from a Server without the Client(s) subscribing to that data or
message, e.g. “Reset”, “Abort”, “Time”. Does not require a
Connection to be established.
Use Case/Scenario
A description of systems functionality. A description of the
sequence of actions that occurs when a user participates in a
dialogue with a system. It describes the behavior that is
invoked by a system function.
Value-Dependent
Operation
An operation in which the behavior of the corresponding
request depends upon which names are used to identify
object parameters (if an object can have multiple names).
Virtual Class (Æ Abstract Class)
Virtual Member
Function
A member function that can be overridden by derived classes
in order to implement a general behavior in a specific
manner. Dynamic binding is used at run time to determine
which of these functions to actually invoke.
263
WAN
Wide Area Network - a communication network which
typically covers a geographical area i.e. The network linking
a Central Control Room to a number of Substations.
Weak Typing
A language characteristic that does not require an explicit
type declaration for each value or expression. Weak typing
makes dynamic binding feasible.
Wrapper
ÆObject wrapper
264
Bibliography
[Alarcón 94]
Alarcón, M.I., Rodríguez, P., Almeida, L.B., Sanz, R., Fontaine, L.,
Gómez, P., Alamán, X., Nordin, P., Bejder, H. and de Pablo, E.;
Heterogeneous Integration Architecture for Intelligent Control.
Intelligent Systems Engineering, Autumn 1994.
[Åstrom 97]
Åström, Karl Johan and Björn Wittenmark (1997). Computer
Controlled Systems. third ed. Prentice-Hall. New York, NJ.
[Åstrom 00]
Åström, Karl, Isidori, Alberto, Albertos, Pedro, Blanke, Mogens,
Schaufel-berger, Walter and Sanz, Ricardo, Eds.) (2000). Control of
Complex Systems. Springer. Berlin.
[Bass 98]
Bass L., Clements P., Kazman R.(1998) Software Architecture In
Practice. 2nd Printing. Addison Wesley. SEI Series in Software
Engineering.
[BBE 02]
Ralf Belschner, Josef Berwanger, Christian Ebner et al.; FlexRay:
Requirements Specification; BMW, DaimlerChrysler, Robert Bosch,
General Motors, and Opel, Germany, 2002.
[Bengtsson 04]
PerOlof Bengtsson, Nico Lassing, Jan Bosch and Hans van Vliet.
Architecture-Level Modifiability Analysis (ALMA). Journal of Systems
and Software, Vol 69, no 1-2, pp 129-147, 2004.
[Binstock 95]
Binstock A., Rex J., (1995) Practical Algorithms for Programmers, First
Printing, Addison Wesley.
[Bittner 03]
Bittner K., Spence I. (2003) Use Case Modeling. Third Printing.
Addison-Wesley.
[Booch 99]
Booch G., Rumbaugh J., Jacobson I. (1999) The Unified Modeling
Language user Guide. Fifth edition. Addison-Wesley
[Borden 95]
Borden, M., E. Crawley, B. Davie and S. Batsell (1995). Integration of
real-time services in an IP-ATM network architecture. RFC 1821. Internet
Engineering Task Force.
[Borland 00]
Inprise, VisiBroker For C++ 4.0 Programmer’s Guide. Inprise
Corporation
265
[Bosch 91]
Robert Bosch GmbH; CAN Specification Version 2.0; Robert Bosch
GmbH, Stuttgart, Germany, 1991.
[Brooks 98]
R. Brooks, C. Breazeal, M. Marjanovic, B. Scassellati, and M.
Williamson. The Cog project: Building a humanoid robot. In Int.
Workshop on Humanoid and Human Friendly Robots, Tsukuba,
Japan, 1998.
[Brown 97]
A.W. Brown and K. Short (1997) “On components and objects: The
foundations of component based development” Proc 5th Int’l Symp.
Assessment of software tools and technologies, IEEE CS Press, Los
Alamitos, Calif., pp. 112-121.
[Brugali 98]
David Brugali. From Objects to Agents: Software Reuse for Distributed
Systems. PhD Thesis. Politecnico di Torino, 1998.
[Burns 97]
A. Burns and A. Wellings (1997) Real-Time Systems and Programming
Languages Addisson-Wesley
[Bus 96]
F. Buschmann, R. Meunier, H. Rohnert, P. Sommerlad, and M. Stal,
Pattern Oriented Software Architecture.
A System of Patterns.
Chichester, UK: Wiley, 1996.
[CAN]
CAN webpage, available at www.can-cia.org
[CDK 94]
George Coulouris, Jean Dollimore, and Tim Kindberg. Distributed
Systems - Concepts and Design. International Computer Science Series.
Addison-Wesley Longman, Inc., 2nd edition, 1994.
[ControlNet]
ControlNet webpage, available at www.controlnet.org
[de Antonio 98] de Antonio A., Arquitectura de Control Inteligente: Un enfoque basado en
componentes para Sistemas de Control, Tesis Doctoral, E.T.S.I.I.
UPM, Madrid, 1998.
[Dev]
DeviceNet webpage, available at www.odva.org
[Dijkstra 68]
E. Dijkstra Cooperating Sequential Processes In Programming
Languages (Genuys F. Ed.), pp. 43-112, Academic Press.
[Doyle 04]
Doyle P, Introduction to Real-time Ethernet I, Contemporary Control
Systems Inc, 2004.
[Doyle 04a]
Doyle P, Introduction to Real-time Ethernet II, Contemporary Control
Systems Inc, 2004.
[Ens 78]
P.H.Enslow What is a distributed data processing system. Computer,
11(1):13-21, January 1978
266
[Foundation]
Foundation Fieldbus webpage, available at www. fieldbus.org
[Gamma 94]
Erich Gamma, Richard Helm, Ralph Jhonson and John Vlissides.
Design Patterns. Elements of Reusable Object Oriented Software.
Addison-Wesley, 1994.
[GE 03]
GE Power Systems, D25 Substation Controller Fact Sheet, General
Electric Substation Automation Solutions, 2003.
[Gokhale 97]
Anirudhha Ghokale, Douglas C. Schmidt, (1997), Evaluating the
Performance of Demultiplexing Strategies for Real-Time CORBA,
GLOBECOM’97, Phoenix, AZ., November 1997
[Hanssen 03]
Hanssen F., Jansen P., Real-time Communications Protocols: an
Overview, IBM Equinox Programme, 2003.
[Har 00]
Florian Hartwich, Bernd Müller, Thomas Führer et al.; CAN
Network with Time Triggered Communication; Robert Bosch
GmbH, Germany, 2000, available at http://www.can.bosch.com/.
[Harsu 02]
Harsu M., A survey on domain engineering. Report 31, Institute of
Software Systems, Tampere University of Technology, December
2002.
[Hashimoto 98] S. Hashimoto et al. Humanoid robots in waseda university – hadaly-2and
wabian. In Int. Workshop on Humanoid and Human Friendly
Robots, Tsukuba, Japan, 1998.
[Helm 92]
Helm, F. v. d., A finite musculoskeletal model of the shoulder mechanism.
Journal of Biomechanics 27(5), 1992.
[HINT 94]
HINT Manual for System Developers. HINT Consortium, 1994.
[Honeywell 04] The Experion Platform, PN-04-016-ENG, Honeywell International
2004.
[Horst 97]
Horst. F. Schaude. Karlsruher Visualisierer. Technical report, Institut
für Prozessrechentechnik und Robotik, Universität Karlsruhe, 1997.
[HRTC 02]
The HRTC Consortium, 2002, HRTC - Annex I – Description of Work
[Hunt 98]
Hunt C., (1998) TCP/IP Network Administration, Second Edition,
O’Reilly & Associates, Inc.
[IEC 01a]
IEC TC-57, Communication networks and systems in substations - part71: Basic communication structure for substation and feeder equipment –
principles and models. Draft Standard IEC 61850-7-1, 2001.
267
[IEC 01b]
IEC TC-57, Communication networks and systems in substations - part72: Basic communication structure for substation and feeder equipment abstract communication service interface (acsi). Draft Standard IEC
61850-7-2, 2001.
[IEC 01c]
IEC TC-57, Communication networks and systems in substations - part1:
Basic principles. Draft Standard IEC 61850-1, 2001.
[Inoue 98]
H. Inoue. A platform-based humanoid robotics project. In Int. Work-shop
on Humanoid and Human Friendly Robots, Tsukuba, Japan, 1998.
[Jalote 94]
Pankaj Jalote. Fault Tolerance in Distributed Systems. Prentice-Hall,
1994.
[John 02]
John I., Muthig D., Modeling Variability with use Cases v1.0, Report
Number 062-02/E, Fraunhofer IESE, November 2002.
[Kapandji 84]
Kapandji, I.A. and Koebpke. Funktionelle Anatomie der Gelenke.
Ferdinand Eneke Verlag, Stuttgart, 1984.
[Lerman 93]
Lerman S., (1993) Problem Solving and Computation for Scientists and
Engineers, Prentice Hall
[Liu 00]
Jane W. S. Liu, (2000) Real-Time Systems, Prentice-Hall
[Martinsson 02] Anders Martinsson, “Scheduling of real-time traffic in a switched
Ethernet network”, M.SC Thesis, Department of Automatic Control,
Lund University, 2002.
[Modbus]
Modbus webpage, available at www.modbus.org
[Mowbray 97]
Thomas J. Mowbray and Raphael C Malveau. CORBA Design
Patterns. Wiley, 1997.
[Nichols 98]
Nichols B., Buttlar D., Proulx J., (1996) Pthreads Programming, First
Edition, O’Reilly
[Ogata 90]
Katsuhiko Ogata. Modern Control Engineering. Prentice Hall, 1990.
[OMG 96a]
An Overview of the OMA. Object management Group.
[OMG 96b]
Realtime CORBA. A White Paper. OMG Realtime SIG. Object
Management Group, 1996.
[OMG 97]
UML Notation Guide, version 1.1 (1 September 1997), The Object
Management Group, doc. no. ad/97-08-05.
[OMG 98a]
Common Object Request Broker Architecture and Specification, Ver.
2.2. Object Management Group, 1998.
268
[OMG 98b]
Real-Time CORBA. Joint Revised Submission Document Number
orbos/1998-12-10, Object Management Group, Needham, MA,
U.S.A.,
December
1998.
Available
at
http://doc.omg.org/orbos/1998-12-10.
[OMG 00]
Concurrency Service Specification v1.0. Document Number
formal/00-06-14, Object Management Group, Needham, MA, U.S.A.,
April, 2000. Available at http://doc.omg.org/formal/00-06-14.
[OMG 00a]
Externalization Service Specification v1.0. Available Specification
Document Number formal/00-06-16, Object Management Group,
Needham,
MA,
U.S.A.,
June
2000.
Available
at
http://doc.omg.org/formal/00-06-16.
[OMG 00b]
Licensing Service Specification v1.0. Available Specification
Needham,
MA,
U.S.A.,
June
2000.
Available
at
[OMG 00c]
Property Service Specification v1.0. Available Specification
Needham,
MA,
U.S.A.,
June
2000.
Available
at
[OMG 00d]
Query Service Specification v1.0. Available Specification Document
Number formal/00-06-23, Object Management Group, Needham,
MA,
U.S.A.,
April
2000.
Available
at
[OMG 00e]
Relationship Service Specification v1.0. Available Specification
Needham,
MA,
U.S.A.,
April
2000.
Available
at
[OMG 00f]
Trading Object Service Specification v1.0. Available Specification
Needham,
MA,
U.S.A.,
June
2000.
Available
at
[OMG 01a]
CORBA 3.0 New Components Chapters. Updated CCM specification
Document Number ptc/2001-11-03, Object Management Group,
Needham, MA, U.S.A., November 30, 2001. Available at
http://doc.omg.org/ptc/2001-11-03.
[OMG 01b]
Real-Time CORBA 2.0: Dynamic Scheduling Specification. Final
Adopted specification Document Number ptc/2001-08-34, Object
269
Management Group, Needham, MA, U.S.A., August 2001. Available
at http://doc.omg.org/ptc/2001-08-34.
[OMG 01c]
Open Communications Interface. Submission to the Extensible
Transport Framework RFP. Document Number orbos/2001-01-05,
Object Management Group, Needham, MA, U.S.A., January 2001.
Available at http://doc.omg.org/orbos/2001-01-05.
[OMG 02]
Common Object Request Broker Architecture and Specification.
Release 3.0. Object Management Group. Falls Church, USA.
[OMG02a]
Enhanced View of Time V1.1. Available Specification Document
MA,
U.S.A.,
May
2002.
Available
at
[OMG 02b]
Fault Tolerant CORBA. Available Specification Document Number
formal/2002-06-59, Object Management Group, Needham, MA,
U.S.A., May 2002. Available at http://doc.omg.org/formal/2002-0659.
[OMG 02c]
Data Acquisition From Industrial Systems v 1.0. Available
Specification Document Number formal/2002-11-07, Object
Management Group, Needham, MA, U.S.A., November 2002.
Available at http://doc.omg.org/formal/2002-11-07.
[OMG 02d]
Utility Management Systems Data Access Facility v 1.0. Available
Specification Document Number formal/2002-11-08, Object
[OMG 02e]
Object Collection Service Specification v 1.0.1. Available
Specification
Document
Number
formal/02-08-03,
Object
[OMG 02f]
Time Service v1.1. Available Specification Document Number
formal/2002-05-06, Object Management Group, Needham, MA,
U.S.A., May 2002. Available at http://doc.omg.org/formal/2002-0506.
[OMG 02g]
Life Cycle Service Specification v 1.2. Available Specification
Needham, MA, U.S.A., September 2002. Available at
270
[OMG 02h]
Naming Service Specification v 1.2. Available Specification
[OMG 02i]
Persistent State Service v 2.0. Available Specification Document
MA,
U.S.A.,
September
2002.
Available
at
[OMG 02j]
Security Service Specification v 1.8. Available Specification
Needham,
MA,
U.S.A.,
March
2002.
Available
at
[OMG 02k]
Minimum CORBA Specification v1.0. Available Specification
Needham,
MA,
U.S.A.,
August
2002.
Available
at
[OMG 02l]
CORBA Components v3.0. Available Specification Document
MA,
U.S.A.,
August
2002.
Available
at
[OMG 02m]
UML Profile for CORBA Specification v1.0. Available Specification
Needham,
MA,
U.S.A.,
April,
2002.
Available
at
[OMG 02n]
Extensible Transport Framework. Joint Revised Submission.
Document Number mars/2002-09-06, Object Management Group,
Needham, MA, U.S.A., September, 2002. Available at
http://doc.omg.org/mars/2002-09-06.
[OMG 03]
Real-time CORBA Specification v2.0. Available Specification
Needham, MA, U.S.A., November, 2003. Available at
[OMG 03a]
Extensible Transport Framework. Revised Submission Document
Number mars/2003-02-01, Object Management Group, Needham,
MA,
U.S.A.,
March
3,
2003.
Available
at
271
[OMG 03b]
Data Distribution Service submission. Joint Submission Document
Number mars/2003-03-16, Object Management Group, Needham,
MA,
U.S.A.,
March,
2003.
Available
at
[OMG 03c]
Smart Transducers Interface V1.0. Available Specification Document
MA,
U.S.A.,
January
2003.
Available
at
[OMG 03d]
Unified Modeling Language V1.5. Available Specification Document
MA,
U.S.A.,
March
2003.
Available
at
[OMG 03e]
Data Distribution Service Final Adopted Specification. Available
Specification
Document
Number
ptc/2003-07-07,
Object
Management Group, Needham, MA, U.S.A., July 2003. Available at
[OMG 03f]
Historical Data Acquisition From Industrial Systems (HDAIS) v 1.0.
Available Specification Document Number dtc/2003-02-01, Object
Management Group, Needham, MA, U.S.A., May 2002. Available at
http://doc.omg.org/dtc/2003-02-01.
[OMG 03g]
Lightweight Logging Service v1.0. Available Specification Document
MA,
U.S.A.,
November
2003.
Available
at
[OMG 03h]
Telecoms Log Service v1.1.1. Available Specification Document
MA, U.S.A., June 2003. Available at http://doc.omg.org/formal/0306-01.
[OMG 03i]
Transaction Service Specification v1.4. Available Specification
[OMG 04]
Event Service Specification v1.2. Available Specification Document
MA,
U.S.A.,
October
2004.
Available
at
272
[OMG 04a]
Lightweight Services Specification v1.0. Available Specification
Needham,
MA,
U.S.A.,
October
2004.
Available
at
[OMG 04b]
Notification Service Specification v1.1. Available Specification
Needham,
MA,
U.S.A.,
October
2004.
Available
at
[OMG 04c]
Lightweight CORBA Component Model. As revised by the
Lightweight CCM 2003 FTF. Document Number ptc/2004-06-10,
Object Management Group, Needham, MA, U.S.A., May 2004.
Available at http://doc.omg.org/ptc/04-06-10.
[OMG 04d]
UML Profile for Modeling Quality of Service and Fault Tolerance
Characteristics and Mechanisms. Final Adopted Specification
Document Number ptc/2004-09-01, Object Management Group,
[OMG 05]
Real-time CORBA Specification v1.2. Available Specification
Needham,
MA,
U.S.A.,
January
2005.
Available
at
[OMG 05a]
UML Profile for Schedulability, Performance and Time Specification
v1.1. Available Specification Document Number formal/05-01-02,
Object Management Group, Needham, MA, U.S.A., January 2005.
[Pancha 93]
Pancha, P. and M. El Zarki (1993). Bandwidth-allocation schemes for
variable-bit-rate mpeg source in atm networks. IEEE Transactions on
Circuits and Systems 3(3), 190–198.
[Pascual 00]
Pascual, J., Charte, F., Segarra, M., de Antonio, A., Clavijo, J.,
Programacióm Avanzada en Windows 2000 con Visual C++ y MFC
McGraw Hill 2000
[Profibus]
Profibus webpage, available at www.profibus.com
[Ravindran 97]
Ravindran, K. and T.J. Gong (1997). Resource allocation control
protocols for multicast data transport. In: IEEE 6th International
Conference on Computer Communications and Networks. Las
Vegas, USA. pp. 22–25.
273
[Rodríguez 99]
Rodríguez Hernández M., Modelado de Sistemas Industriales Complejos,
Tesis Doctoral, E.T.S.I.I., UPM, Mayo, 1999.
[Sanz 00]
Sanz, Ricardo (2000). Agents for complex control systems. Chap. 10,
pp. 171–190. In: Automation, Control and Complexity. an Integrated
Approach, Eds. Samad and Weyrauch, John Wiley & Sons. 2000.
[Sanz 91]
Sanz, R., A.Jiménez, R.Galán, F.Matía and E.A.Puente. Intelligent
Process Control: The CONEX Architecture. In Engineering Systems
with Intelligence. S. Tzafestas (Ed.). Kluwer Academic Publishers,
1991.
[Sanz 94]
Sanz, R., R.Galán, A.Jiménez, F.Matía, J.Velasco and G.Martínez.
Computational Intelligence in Process Control. ICNN'94, IEEE
International Conference in Neural Networks. Orlando, USA, 1994.
[Sanz 96]
Sanz, R., F.Matía, R.Galán and A. Jiménez. Integration of Fuzzy
Technology in Complex Process Control Systems. FLAMOC'96.
Sydney, Australia, 1996.
[Sanz 99a]
Sanz, Ricardo, Idoia Alarcón, Miguel J. Segarra, Angel de Antonio
and José A. Clavijo (1999a). Progressive domain focalization in
intelligent control systems. Control Engineering Practice 7(5), 665–
671.
[Sanz 99b]
Sanz, Ricardo, Miguel J. Segarra, Angel de Antonio and José A.
Clavijo (1999b). ICa: Middleware for intelligent process control. In:
IEEE International Symposium on Intelligent Control, ISIC’1999.
Cambridge, USA.
[Sanz 99c]
Sanz, Ricardo, Miguel J. Segarra, Angel de Antonio, Fernando Matía,
Agustín Jiménez and Ramón Galán (1999c). Patterns in intelligent
control systems. In: Proceedings of IFAC 14th World Congress.
Beijing, China.
[Sanz 03]
Sanz, Ricardo and Janusz Zalewsky. Control Engineering using
Design Patterns. IEEE Control Systems Magazine, June 2003.
[Sanz 03a]
Sanz R., Yela A., Chichilla R., A Pattern Schema for Complex
Controllers, 9th IEEE International Conference in Emerging
Technologies and Factory Automation, Lisbon, Portugal, September
2003.
[Seckin 98]
Seckin, Gamze and Forouzan Golshani (1998). A new model for
multilayer real-time video transmission for ATM networks. In:
274
Proceedings of the 7th International Conference on Computer
Communications and Networks. Lafayette, USA.
[Selic 94]
Bran Selic, Garth Gullekson and Paul T. Ward. Real-Time Object
Oriented Modelling. Wiley, 1994.
[Selic 96]
Bran Selic & Garth Gullekson. Design Patterns for Real-Time
Software. Embedded Systems Conference West ’96. 1996.
[Selic 98]
Bran Selic and Jim Rumbaugh. Using UML for Modeling Complex
Real-Time Systems. March 11, 1998
[Schmidt 95]
Douglas C. Schmidt, 1995, Using Design Patterns to Develop Reusable
Object-Oriented Communication Software, CACM October’95
[Schmidt 98]
Douglas C. Schmidt, David L. Levine and Chris Cleeland, 1998,
Architectures and Patterns for Developing High-Performance, Real-Time
ORB Endsystems.
[Schmidt 97]
Schmidt D. C. (1997) Acceptor and Connector: Design Patterns for
Initializing Communication Services in Pattern Languages of Program
Design (R. Martin, F. Buschmann, and D. Riehle, eds.), Reading, MA:
Addison-Wesley.
[Schmidt 00]
D. Schmidt and F. Kuhns, An Overview of the Real-Time CORBA
Specification, IEEE Computer, June 2000
[Sha 93]
Sha, L. and Sathaye, S.S. A Systematic Approach to Designing
Distributed Real-Time Systems. IEEE Computer, Vol.26, No.9, 1993.
[Sha 95]
Sha, Lui; Rajkumar, Ragunathan; Gagliardi, Michael. A Software
Architecture for Dependable and Evolvable Industrial Computing
[Sheldon 95]
Sheldon T., (1995) LAN Times Guide to Interoperability, First Edition,
Osborne McGraw-Hill.
[Sommerville 95] Sommerville I., (1995) Software Engineering, Fifth edition. AddisonWesley
[Stroustrup 90]
Stroustrup B., Ellis M., (1990) The Annotated C++ Reference Manual,
Addison-Wesley
[Stroustrup 97]
Stroustrup B., (1997) The C++ Programming Language, 3rd Edition,
Addison Wesley.
[Stallings 97]
Stallings W., (1997) Data & Computer Communications, 5th Edition.
Prentice Hall, Inc.
275
[Tanenbaum 95] Andrew S. Tanenbaum. Distributed Operating Systems. Prentice Hall,
Inc., Englewood Cliffs, New Jersey 07632, 1995.
[Tanenbaum 97] Tanenbaum C., (1997) Computer Networks, 3rd Printing. Prentice
Hall, Inc.
[Tanenbaum 97a] Tanenbaum A., Woodhull A., (1997) Operating Systems Design and
Implementation, Second Edition, Prentice-Hall International, Inc.
[Temple 03]
Temple C., FlexRay Protocol Overview, FlexRay 4th International
Workshop, Detroit, U.S.A., March 2003.
[Trigaux 03]
Trigaux JC., Heymans P. (2003) Modelling variability requirements in
software product lines: A comparative survey. PLENTY Project.
[TTP 02]
TTTech, Time Triggered Protocol TTP/C High Level Specification
Document, Specification Edition 1.0.0, 4th July 2002
[TTTech 05]
TTech, TTP by-wire Box-Automotive Control Solution for Brushless DC
Motors, TTTech Computertechnik AG, 2005.
[Wooldridge 00] Michael Wooldridge, Nicholas R. Jennings and David Kinny. The
Gaia Methodology for Agent-Oriented Analysis and Design.
Autonomous Agents and Multi-Agent Systems, 3, 285-312, 2000.
[World]
World FIP webpage, available at www.worldfip.org
276
277

CORBA Control Systems - UPM ASLab

Transcripción

Documentos relacionados

Sts. Peter and Paul Catholic Church Sanctuary of St. Toribio Romo