CORBA Control Systems - UPM ASLab
Transcripción
CORBA Control Systems - UPM ASLab
T HE A SYSTEMS LABORATORY www.aslab.org UTONOMOUS CORBA Control Systems A Distributed Object Approach to Complex Control Engineering Miguel José Segarra R-2005-012 2005-7-19 UNIVERSIDAD POLITÉCNICA DE MADRID ESCUELA TÉCNICA SUPERIOR DE INGENIEROS INDUSTRIALES CORBA CONTROL SYSTEMS PhD Thesis Miguel José Segarra Martínez Ingeniero Industrial 2005 This page has been intentionally left blank UNIVERSIDAD POLITÉCNICA DE MADRID ESCUELA TÉCNICA SUPERIOR DE INGENIEROS INDUSTRIALES Departamento de Automática, Ingeniería Electrónica e Informática Industrial CORBA CONTROL SYSTEMS PhD Thesis Author: Director: MIGUEL JOSÉ SEGARRA MARTÍNEZ Ingeniero Industrial RICARDO SANZ BRAVO Doctor Ingeniero Industrial Madrid, 2005 This page has been intentionally left blank Título: CORBA CONTROL SYSTEMS Autor: MIGUEL JOSÉ SEGARRA MARTÍNEZ Ingeniero Industrial TRIBUNAL Tribunal nombrado por el Mgfco. y Excmo. Sr. Rector de la Universidad Politécnica de Madrid el día 22 de Junio de 2005. Presidente: Dr. Ramón Galán López Vocales: Dr. Santos Galán Casado Dra. Miren Idoia Alarcón Rodríguez Dr. Ángel de Antonio Bermejo Secretario: Dr. Manuel Rodríguez Hernández Suplentes: Dr. Juan Ramón Velasco Pérez Dra. Marga Marcos Muñoz Realizado el acto de la lectura y defensa de la Tesis el día 19 de Julio de 2005. Calificación de la Tesis...................... El Presidente El Secretario Los Vocales This page has been intentionally left blank For Cloé and Hugo. Para Cloé y Hugo. Pour Cloé et Hugo. ― Miguel This page has been intentionally left blank Acknowledgments This has always looked to me one of the hardest parts to write of any book. It always seems there is not a fine way to thank all the people who have worked with me in many different areas over the last years. For sure, some of you will not find your name here. It does mean neither that you have not contributed to what I know nor that you have not contributed to this Thesis in one way or another. Simply, I cannot recall all the important things many people have taught me and that have been used in my work. Consider yourself acknowledged in these lines if your name just slipped from my mind. Of course, I am acknowledging the people I can remember. I will start with my co-workers. José Antonio Clavijo, my partner at SCILabs Ingenieros, has spent the last few years with me trying to build up the business. It is tough thing to do and we have spent many days working around the clock (literally) in order to hit the deadline. We never had a serious complaint from a customer and we are proud our software is involved in the construction of some of the largest infrastructures of Spain. We have built together the new version of the ICa ORB. José Antonio, your turn is next for showing up your Thesis work. Carlos Moreno is another co-worker at SCILabs that recently left to work in the Telecom industry. In spite of that, we are friends and he has been with José Antonio and me from the very beginning. He has also worked very hard with us and knows well the late hours of night at the office. I would have liked he could remain with us at SCILabs. Eventually, he became very pushy with me about finishing this Thesis work which I appreciate. Angel de Antonio, Associate Professor of the Universidad Autónoma de Madrid, developed the first version of ICa. That first ORB, without being fully CORBA compliant but following its spirit, was a large leap from the previous communications software being employed at ASLAB. I spent with him many hours and days testing and debugging many pieces of that initial software. That ORB was the precursor of what the ICa ORB is today. Probably, the ICa ORB would not be what it currently is without that initial work. I have to thank him for the good job. The European Union IST programme has a lot to do with the funding of this work. I possibly could not finish this Thesis without its support. I think the help received from the EU has not only been economic. Behind funding and bureaucracy there are people that might believe in what you are doing or that might see an opportunity for the technical, social and economical development of Europe. Consequently, I would like to thank these people I have found abroad and that have certainly contributed to CORBA Control Systems. First, from the European Commission, I want to thank Alkis Konstantellos who is part of the Embedded Systems Unit of IST. When SCILabs spun off I already had some background in the CORBA world from my previous work at ASLab-UPM (mainly from the DIXIT and Hydra projects). By that time, a Spanish PLC manufacturer and engineering firm, ELIOP, contacted SCILabs to participate in an IST project where CORBA technology was involved. The project was DOTS (more on DOTS can be found in the Thesis). This project, which SCILabs’ part I directed, was the first international consortium of the firm. I remember the first review meeting in Rome (Italy) was highly stressing for me. There were lots of things that could go wrong as we had just started the company and had neither a good financial support nor business background experience to prove our capabilities. However, it turned out that Dr. Konstantellos, at that moment the project officer from the EU, preferred to make decisions based on facts and I had my chance to prove what SCILabs could do. The real-time ICa ORB was developed during the DOTS project. Alkis Konstantellos trusted SCILabs and it did not deceive him. He also deserves acknowledgement in these lines. I would also like to thank Professor Hermann Härtig, one of the DOTS project reviewers. At the beginning of the project, he was not sure that SCILabs could meet the deadline. He openly said so in the DOTS first review meeting held in Rome (Italy) which I refuted by exposing that all the tasks were carefully planned and scheduled. By the end of the project, he was greatly surprised by the amount of work achieved. Later, after the end of DOTS, he invited me as a speaker to the SDA 2002 (System Design Automation Workshop) at the Technical University of Dresden (Germany), where I spent a wonderful time with the people at the Computer Science department. He was very helpful in all reviews and tried to supply positive ideas and recommendations for the ongoing work. Also, after the project, he offered me to integrate the ICa ORB with the Fiasco Microkernel being developed at Dresden Technical University. At the time of the offer, I could not spare the needed resources to do it but I do not forget his offering and maybe it is possible in the future. I want to show my appreciation for the work carried out in DOTS by some of ELIOP’s technical staff. The hardware for the new RTUs of ELIOP was also developed during the project which put additional stress (and risk) into the schedule as the machine and the broker were new and, both had to be integrated with the application software. Ignacio González and Fabrice Wawak did a great work to solve many problems related to hardware and basic software of the RTU. I hope both of you have already forgotten that little code bug in the Ethernet chip of the CNI. Also, Francisco Rodríguez did a great job as overall project coordinator of DOTS by constantly remembering everybody their short term duties. The work done in the DOTS project grew the confidence of the European Commission in the possibilities of ICa for the distributed control purpose and, an ambitious project was prepared, Hard Real-Time CORBA, in which SCILabs, UPM and two worldleading universities in Computing and Control theory, the University of Lund (Sweden) and the Technical University of Vienna (Austria) participated. From the University of Lund, I want to acknowledge Professor Åström. I had the opportunity to show him the work being done in our team at ASLAB when he came to Madrid for the homage to Professor Puente. He said that our team at ASLAB was “very aggressive with work” which is very nice to hear from someone so respected. From this visit, Professor Sanz kept in contact with the people of Lund University and thus, after several years and interim activities with them, we were able to launch a joint project. Also I would like to mention Professor Årzén and Associate Professor Nilsson from Lund University, management colleagues of the HRTC project, for their valuable contributions and insights regarding networked control systems. From the Technical University of Vienna, I want to recognise the contribution of Professor Kopetz, the man behind TTP. Once he is convinced of his ideas, he defends them in a very sternly and stubborn way. Having him in the project always stirred the pot and there were interesting discussions in project meetings about linking interfaces, temporal firewalls and many other things. Also Thomas Losert, a PhD student of the Technical University of Vienna, helped in the understanding of TTP and on the possible mapping to a CORBA transport plug-in. The HRTC project also counted with two good reviewers. First, Professor Thomson from the University of Sheffield made interesting contributions and comments on the work of HRTC. Professor Thomson has a closed English accent leading to some funny situations during the project for non-UK citizens. I will make a proposal in my next international project for non-UK people to take a British English course so we can speak and understand real English. Secondly, Dr. Virginie Watine from Thales Communications who has a deep understanding of CORBA and is very involved with OMG. She clearly understood that the ICa ORB is the only ORB in Europe (probably in the world) dealing with the problem of hard real-time. I also thank her for inviting me, after HRTC concluded, to Thales in Paris (France) to the initial meetings for the preparation of a project on CORBA components. From UPM I would like to thank Professor Ricardo Sanz. It is many years now that we know each other and we have developed a bond of friendship. He has helped me enormously either academically or in my professional career. Certainly, it would be impossible to provide here a list of the many things he has taught me or in which he provided direction to me. Also, I thank him for counting on SCILabs for the development of international research projects and I am happy we have reached a framework of agreement for the research use of ICa by his team at ASLab. Of many things, I would like to point out the extreme availability of Ricardo to whatever the request I made him. Really, he puts up with a lot of different things at the same time and, as you know, it is difficult to maintain a high level of context switching and be productive. I can certify he is. Just one tip of advice for him, Ricardo, look at your travel agenda, you cannot get into more things! I would like also to give thanks to my family. They always support me whatever the decision I make. My deepest gratitude to my parents, Miguel and Francisca, for giving me the opportunity of having a good education. Also thanks to my sisters, Carmen and Sonia, and to Cloé and Hugo, Sonia’s little daughter and son that make life joyful when they come from Paris. Last, but not least, my girlfriend (I am not married and cannot find a better word) Rosanna pushed me to go on when my spirit ran short and morale went low. I thank her for grounding me in reality, for her endurance and persistence in making me finish this work and, most of all, for being my friend. Miguel José Segarra Martínez Mayo de 2005 Agradecimientos Esta siempre me ha parecido una de las partes más difíciles de escribir de cualquier libro. Siempre parece que no existe una buena manera de dar las gracias a toda la gente que ha trabajado conmigo en muchas áreas diferentes durante los últimos años. A buen seguro, algunas de esas personas no encontrarán su nombre en estas líneas. Eso no quiere decir que no hayan contribuido a lo que sé o que no hayan contribuido a la realización de esta Tesis de una manera u otra. Sencillamente, no me resulta posible recordar todas las cosas importantes que me habéis enseñado y que se han utilizado en mi trabajo. Considerad vuestra aportación reconocida en estos agradecimientos si olvido mencionaros. Por supuesto, voy a dar las gracias a las personas que sí recuerdo. Empezaré con mis compañeros de trabajo. José Antonio Clavijo, mi socio en SCILabs Ingenieros, ha pasado los últimos años conmigo tratando de sacar adelante la empresa. Es algo verdaderamente difícil de hacer y hemos pasado muchos días enteros trabajando (literalmente) para cumplir con los compromisos que habíamos adquirido. Nunca hemos tenido una queja seria de ningún cliente y estamos orgullosos de que nuestro software sirva para la construcción de algunas de las infraestructuras más grandes de España. Hemos construido juntos la nueva versión del ORB ICa. José Antonio, tu turno es el próximo para presentar tu trabajo de Tesis Doctoral. Carlos Moreno es otro trabajador de SCILabs que recientemente nos dejó para trabajar en la industria de las Telecomunicaciones. A pesar de esto, somos amigos y ha estado con José Antonio y conmigo desde el principio. También ha trabajado muy duro y conoce perfectamente las horas tardías de la noche en la oficina. Me hubiese gustado que siguieras con nosotros en SCILabs. Desde hace unos meses ha sido muy insistente para que terminara esta Tesis, cosa que le agradezco. Ángel de Antonio, Profesor Asociado de la Universidad Autónoma de Madrid, desarrolló la primera versión del broker ICa. Ese primer ORB, que aunque no cumplía completamente con la especificación de CORBA seguía su espíritu, fue un gran salto adelante en comparación con el software de comunicaciones que usábamos previamente en ASLab. Empleé muchas horas y días con él, probando y depurando muchos componentes de aquel software inicial. Aquel ORB fue el precursor de lo que ICa ORB es hoy. Probablemente, la versión actual de ICa ORB no existiría sin aquel desarrollo inicial. Le agradezco que realizara un buen trabajo. El programa IST de la Unión Europea tiene mucho que ver con la financiación de este trabajo. Probablemente, no hubiera podido acabar esta Tesis sin su apoyo. Creo que el apoyo de la Unión Europea no ha sido sólo económico. Detrás de la financiación y la burocracia hay personas que pueden creer en lo que uno hace o que pueden ver una oportunidad para el desarrollo técnico, social o económico de Europa. Consecuentemente, me gustaría dar las gracias a esas personas que he encontrado fuera de mi país y que, ciertamente, han contribuido a los Sistemas de Control CORBA. Primero, perteneciente a la Comisión Europea, me gustaría agradecer su apoyo a Alkis Konstantellos, quien forma parte de la Unidad de Sistemas Empotrados del programa IST. Cuando SCILabs surgió como empresa, yo ya tenía experiencia en el mundo de CORBA a través de mi trabajo previo en ASLab-UPM (fundamentalmente de los proyectos DIXIT e Hydra). Por esa época, un fabricante español de PLCs se puso en contacto con SCILabs para ofrecernos participar en un proyecto IST donde la tecnología CORBA era necesaria. El proyecto era DOTS (más sobre DOTS se puede encontrar en la Tesis). Este proyecto, en el que dirigí la participación de SCILabs, fue el primer consorcio internacional de la empresa. Recuerdo que la primera reunión de revisión del proyecto en Roma (Italia) fue muy estresante para mí. Había muchas cosas que podían ir mal ya que prácticamente habíamos comenzado nuestra andadura como compañía y no contábamos ni con un buen soporte financiero ni con años de experiencia empresarial que permitieran demostrar nuestra capacidad. Sin embargo, ocurrió que el Dr. Konstantellos, en aquel momento el oficial de la Comisión Europea a cargo del proyecto, prefirió tomar decisiones basándose en hechos y tuve la oportunidad de demostrar lo que SCILabs podía hacer. El ORB de tiempo real de ICa fue desarrollado a lo largo del proyecto DOTS. Alkis Konstantellos confió en SCILabs y ésta no le decepcionó. El también merece mi agradecimiento en estas líneas. También quiero dar las gracias al Profesor Hermann Härtig, uno de los revisores nombrados por la Comisión Europea para proyecto DOTS. Al comienzo de DOTS, él no estaba convencido de que SCILabs pudiera cumplir los plazos del proyecto. Lo comentó abiertamente en la primera revisión del proyecto en Roma (Italia) y yo le refuté exponiendo que todas las tareas estaban cuidadosamente organizadas y planificadas. Al final del proyecto, estaba absolutamente sorprendido de todo el trabajo realizado. Más tarde, tras la finalización del proyecto DOTS, me invitó a dar una conferencia en el SDA 2002 (System Design Automation workshop) en la Universidad Técnica de Dresde (Alemania), donde pasé unos días realmente agradables con los profesores y estudiantes de postgrado del Departamento de Ciencias de la Computación. Hermann Härtig supo ser de utilidad en todas las revisiones del proyecto DOTS, tratando siempre de proporcionar ideas y recomendaciones positivas para el trabajo en desarrollo. También, después del proyecto DOTS, me ofreció la posibilidad de integrar el ORB ICa con el microkernel Fiasco que se estaba desarrollando en la Universidad Técnica de Dresde. En aquel momento tuve que desechar la oferta ya que no disponía de los recursos necesarios para hacerlo. Quizás sea posible en el futuro. También quiero mostrar mi aprecio al personal técnico de ELIOP por el trabajo realizado en DOTS. El hardware de las nuevas RTUs de ELIOP se desarrolló también durante el proyecto, lo que introdujo presión y riesgo adicional a su planificación ya que la máquina utilizada e ICa ORB eran nuevos, y ambos debían integrarse con el software de aplicación. Ignacio González y Fabrice Wawak hicieron un gran trabajo para resolver muchos problemas relacionados con el hardware y el software básico de las RTU. Espero que ya os hayáis olvidado de aquel pequeño bug del código del chip Ethernet del CNI. Agradezco también a Francisco Rodríguez su gran trabajo como coordinador general de DOTS, recordando insistentemente a todos la cercanía del cumplimiento de nuestras obligaciones más inmediatas. El trabajo realizado en el proyecto DOTS hizo que creciera la confianza que la Comisión Europea depositó en las posibilidades de ICa para su uso en sistemas de control distribuido y, en consecuencia, se preparó un proyecto ambicioso, Hard RealTime CORBA, en el que SCILabs, la UPM y dos Universidades líderes mundiales en ciencias de la computación y teoría de control, la Universidad de Lund (Suecia) y la Universidad Técnica de Viena (Austria) participaron. De la Universidad de Lund quiero mostrar mi agradecimiento al Profesor Åström. Tuve la oportunidad de mostrarle el trabajo que llevábamos a cabo en nuestro grupo de trabajo en ASLab cuando vino a Madrid para participar en el homenaje al Profesor Puente. Comentó que nuestro grupo “era muy agresivo con el trabajo” lo que sin duda es muy agradable escuchar cuando viene de alguien tan respetado. A partir de esta visita, el Profesor Sanz mantuvo el contacto con los equipos de investigación de Lund y de esta manera, tras varios años y diversas actividades intermedias con ellos, surgió la posibilidad de lanzar un proyecto conjunto. También me gustaría mencionar al Profesor Årzén y al Profesor Nilsson, colegas de gestión del proyecto HRTC, por sus valiosas contribuciones y puntos de vista sobre los sistemas de control en red. De la Universidad Técnica de Viena, quiero agradecer la contribución del Profesor Kopetz, el hombre tras el protocolo TTP y la arquitectura TTA. Una vez está convencido de sus ideas las defiende de una manera tenaz, incluso obstinada. Su participación en el proyecto HRTC siempre fue un estímulo y hubo discusiones interesantes en las reuniones del proyecto sobre interfases de enlace, cortafuegos temporales y muchas otras cosas. También Thomas Losert, un estudiante de postgrado de la Universidad Técnica de Viena, fue de ayuda en la comprensión del protocolo TTP y en su posible implementación como un plug-in de transporte para ICa ORB. El proyecto HRTC también contó con dos buenos revisores de la Unión Europea. Primero, el Profesor Thomson de la Universidad de Sheffield realizó contribuciones y comentarios interesantes sobre el trabajo realizado en HRTC. El Profesor Thomson tiene un cerrado acento Inglés, lo que produjo algunas situaciones divertidas durante el proyecto HRTC para los no Británicos. En mi próximo proyecto internacional haré una propuesta para que los no pertenecientes al Reino Unido tomemos un curso de Inglés Británico. De esta manera podremos hablar y entender el auténtico idioma Inglés. En segundo lugar, la Dra. Virginie Watine de Thales Communications que posee un profundo conocimiento de CORBA y está muy involucrada en las tareas del OMG. Claramente comprendió que ICa es el único ORB en Europa (y probablemente en el mundo) que ataca el problema del tiempo real crítico. Le agradezco que me invitara, al concluir el proyecto HRTC, a Thales en París (Francia) para participar en las reuniones preparatorias de un proyecto sobre componentes CORBA. De la Universidad Politécnica de Madrid quiero dar las gracias al Profesor Titular Ricardo Sanz. Son muchos años ya los que nos conocemos y hemos desarrollado un lazo de amistad. Me ha ayudado y aconsejado enormemente tanto académicamente como en mi carrera profesional. Sería imposible proporcionar aquí una lista de las muchas cosas que me ha enseñado o a las que ha proporcionado dirección. También le agradezco que haya contado con la participación de SCILabs para el desarrollo de proyectos internacionales de investigación y me alegra que hayamos llegado a un acuerdo para el uso de ICaORB por su equipo UPM-ASLab para proyectos de investigación. Entre muchas cosas, me gustaría destacar la extrema disponibilidad de Ricardo ante cualquier petición que le realizara. En realidad, él lleva una multitud de asuntos de manera simultánea y, como sabéis, es difícil mantener un grado elevado de cambios de contexto y ser productivo al mismo tiempo. Puedo decir que él lo es. Sólo un pequeño consejo para él. Ricardo, mira tus viajes en la agenda - ¡No puedes meterte en ninguna cosa más! También quiero dar las gracias a mi familia. Siempre me apoyan tome la decisión que tome. Mi más profunda gratitud a mis padres, Miguel y Francisca, por darme la oportunidad de disfrutar de una buena educación. También doy las gracias a mis hermanas, Carmen y Sonia, y a Cloé y Hugo, los pequeños hija e hijo de mi hermana Sonia que hacen la vida más gozosa cada vez que vienen de Paris. Por último, y desde luego no el menor, quiero dar mi agradecimiento a mi novia (no estoy casado y no encuentro una palabra mejor), Rosanna, que me ha empujado a seguir cuando fallaba el espíritu y la moral estaba baja. Le agradezco el ponerme los pies en el suelo, su resistencia, persistencia y paciencia para conseguir que acabe este trabajo y, sobre todo, que sea mi amiga. Miguel José Segarra Martínez Mayo de 2005 Resumen Hoy en día la mayor parte de los sistemas de control están compuestos por otros sistemas que forman un todo heterogéneo en un entorno distribuido. Son sistemas complejos en los que partes de cada uno de los sistemas-componente se combina con el resto para proporcionar funciones globales. La heterogeneidad surge porque existen equipos especializados que realizan mejor ciertas tareas del sistema: lectura de sensores o control de actuadores, gestión del proceso o gestión empresarial se sitúan en diferentes niveles del sistema de control distribuido y realizan diferentes tareas. En el caso de los sistemas de control, la razón principal para la distribución del sistema es que el proceso de cálculo del algoritmo de control se sitúe cerca del proceso. De esta manera, se asegura una reacción temprana ante los cambios del mismo. Un ejemplo de este tipo de grandes sistemas de control distribuido son las plantas de proceso, cuyo máximo exponente pueda ser quizás la industria petroquímica. También existen otras razones que hacen que los sistemas de control sean distribuidos. • • • • • • • • Disponibilidad de procesadores para sistemas empotrados. Requisitos temporales que hacen prohibitiva la utilización de comunicaciones debido a las latencias. La necesidad de proporcionar un elevado rendimiento del sistema por lo que se recurre a la utilización de paralelismo en la computación. La simplificación de las tareas de construcción y mantenimiento mediante la utilización del concepto de modularidad. La reducción del coste y tiempo de puesta en el mercado mediante la reutilización de componentes. La integración con sistemas propietarios existentes en la planta. La disponibilidad de plataformas software específicas para algunos propósitos. Fiabilidad y tolerancia a fallos. El ejemplo de la planta de proceso se puede trasladar fácilmente a otros dominios en los que la heterogeneidad y distribución de los sistemas aparecen de manera natural. El lector puede pensar en los modernos sistemas de combate donde la información fluye desde muchas fuentes diferentes (radar, satélites, sistemas de adquisición de blancos, etc.) y es recogida por una cantidad similar de sistemas sumidero (tropas, aviónica, sistemas de posicionamiento, etc.). Otro ejemplo se encuentra en la industria del automóvil. Los modernos sistemas de ABS tienen un computador dedicado al control de cada rueda que intercambia la información de frenado y deslizamiento con el resto. Además, la suspensión y el sistema de gestión del motor colaboran con el ABS para mantener el automóvil bajo control. Aunque estos sistemas son comunes en la actualidad, presentan la característica de que son difíciles de construir. Mientras que el coste del hardware decrece a lo largo del tiempo, el software que se ejecuta sobre él se hace cada vez más complejo y el esfuerzo y capital necesarios para construirlo continúa aumentando. La heterogeneidad y distribución de los sistemas contribuyen en gran medida al incremento de los costes y a la dificultad asociados a la construcción del software. Por ejemplo, la construcción del software supone el 50% del coste del nuevo Airbus 380. En esta Tesis se ha realizado una investigación sobre la tecnología CORBA (Common Object Request Broker Architecture) para atacar los problemas descritos en el campo de los sistemas distribuidos de control. Es un campo amplio. Los sistemas de control varían desde los muy pequeños, empotrados y de recursos limitados, hasta los muy grandes, que abarcan una o varias plantas de proceso y contemplan desde el control del proceso en sí mismo hasta la gestión de los pedidos a los proveedores. Sus requisitos también varían del mismo modo, desde los temporalmente críticos hasta aquellos en los que el tiempo en el que ocurren las acciones de control no es el condicionante primordial. En una situación ideal, la mejor manera de solventar los problemas de estos sistemas es sencillamente hacerlos desaparecer en la etapa de construcción del sistema de control. Para conseguir ese propósito se necesita una abstracción común a una amplia gama de sistemas de control. La abstracción común debe ser un mecanismo independiente de las plataformas de manera que la heterogeneidad pueda ser eliminada del sistema. Además, para prescindir de la complejidad impuesta por la distribución, el sistema debería ser desarrollado como si fuera un sistema monolítico. Estas son las ideas que subyacen tras la OMA (Object Management Architecture). La Arquitectura de Gestión de Objetos proporciona una abstracción sobre los problemas de bajo nivel inherentes a cada plataforma y proporciona transparencia a los sistemas en cuanto a la distribución y heterogeneidad de una manera orientada a objetos. La concreción de esa abstracción se consigue mediante un modelo de objetos basados en la OMA. Un ejemplo de esta concreción es la especificación CORBA. Las implementaciones de la especificación CORBA proporcionan a los desarrolladores de sistemas una infraestructura homogénea e independiente de la plataforma para la realización de su trabajo. Mientras que la abstracción de CORBA ha probado su utilidad en algunos dominios de aplicación, el modelo no es lo suficientemente bueno en algunas situaciones. CORBA se diseñó con los objetivos de flexibilidad, interoperabilidad y reducción de la complejidad sin incurrir en una pérdida excesiva de rendimiento. Desafortunadamente, existen requisitos en algunas aplicaciones, como los sistemas de control, respecto a las características de tiempo real, tolerancia a fallos o capacidad para despliegue en sistemas embarcados o empotrados que no pueden dejarse de lado si existe la pretensión de que el sistema opere en el mundo real. Se hace necesario pues un análisis y una implementación de la infraestructura necesaria para el desarrollo de este tipo de sistemas. Como base fundamental para este trabajo, y dada la tecnología elegida como infraestructura (CORBA), se utiliza la Arquitectura Software como elemento de referencia a la hora de modelar sistemas que sean capaces de cumplir con los requerimientos que de ellos se exigirán. La tecnología de objetos y objetos distribuidos se utiliza como medio para realizar representaciones abstractas del mundo y como infraestructura de comunicación entre los mismos. La Arquitectura del Software es importante porque aparece en las primeras etapas del ciclo de vida del sistema y ayuda a tomar decisiones sobre su composición y estructura, y sobre las relaciones existentes entre los elementos del mismo. La definición de una arquitectura software ayuda a establecer compromisos entre las capacidades técnicas que disfrutará un sistema determinado. De aquí la importancia del uso de arquitecturas sólidas para el diseño de sistemas de control basados en CORBA. La tecnología de objetos es importante porque permite expresar de una manera sencilla la arquitectura software de los sistemas. Los objetos encapsulan su estado y proporcionan una interfaz para acceder al mismo. Los objetos también ayudan a capturar las entidades esenciales de cualquier sistema y sus propiedades, permitiendo dividir el sistema en partes que son manejables por las personas. Cuando a la tecnología de objetos le añadimos la capacidad de que éstos se encuentren distribuidos en diferentes piezas hardware o software mediante una infraestructura que les permite colaborar de una manera natural en el sistema distribuido, disponemos de una herramienta que permite construir sistemas complejos, heterogéneos y distribuidos de una manera sencilla y rápida. Con estas ideas en mente se desarrolla esta Tesis doctoral en la que se persiguen dos objetivos fundamentales. • • La investigación de la aplicación de la tecnología CORBA para la resolución de los problemas de control asociados a la heterogeneidad, distribución, complejidad, escalabilidad y falta de rendimiento de los sistemas actuales basados en CORBA. Este trabajo se ha realizado dentro del marco de los grandes sistemas industriales así como en sistemas empotrados de recursos reducidos. Se han construido algunas extensiones a las especificaciones de CORBA para tiempo real y se ha co-desarrollado un broker con el propósito exclusivo del control industrial. Como validación de los trabajos realizados existen numerosos ejemplos de aplicación desarrollados en los últimos años. El segundo objetivo es el desarrollo de una metodología para la construcción de sistemas de control distribuido basados en ICa ORB, el broker CORBA de tiempo real en el que se basan los trabajos aquí expuestos. El propósito de la metodología es la reducción del tiempo y esfuerzo dedicados a la construcción de este tipo de sistemas. La Tesis se encuentra organizada en ocho capítulos. El primer capítulo sirve de introducción. Contiene información sobre las propiedades que exhiben los sistemas de control distribuido. Como se ha comentado en las páginas anteriores la heterogeneidad, complejidad y la necesidad de integración se encuentran entre los factores más importantes a considerar. El segundo capítulo, Sistemas de Control Industrial y Protocolos de Comunicación, realiza un repaso del estado del arte de los sistemas distribuidos de control que se encuentran disponibles de manera comercial. También se realiza un repaso de los protocolos de comunicación más utilizados en la industria de control. Como se ha indicado, el principal problema de la construcción de este tipo de sistemas es la complejidad. La dificultad de dar solución al problema de control en sí mismo se ve aumentada por el hecho de construir un sistema distribuido e intensivo en software. El segundo gran problema es el de la integración. El sistema de control distribuido no puede ser cerrado, multitud de sistemas diferentes deben coexistir. La información debe fluir a través de todos los subsistemas de la planta sea cual sea su objetivo: operacional, táctico o estratégico. La integración siempre resulta difícil y costosa e incluso son los propios fabricantes los que ponen barreras tecnológicas con el fin de promocionar sus propios productos. La Arquitectura de Control Integrado (ICa) soluciona este problema proporcionando un marco de integración común a todos los sistemas que intervienen en el control. El tercer gran problema es el de la flexibilidad. Muchos sistemas de control se construyen a partir de un diseño ad-hoc, específico para el problema a solucionar en un momento dado del tiempo y que imponen una estructura rígida a la forma en que se opera el proceso. También es necesario considerar el futuro y las cambiantes condiciones del mercado, donde será necesario adaptar el entorno tan pronto como sea posible a nuevas necesidades de producción y siempre antes que la competencia. La capacidad de adaptación del sistema incurrirá en mayor o menor medida a los costes de evolución de la planta. Tecnologías como ICa permiten reducir el esfuerzo necesario para la adaptación de los sistemas que deben evolucionar o cambiar. El cuarto gran problema es el de la estandarización. Por ejemplo, los fabricantes de sistemas de automatización hacen grandes esfuerzos para distinguir las interfases de sus productos y tratan de imponer sus “estándares” de tecnología lo que conlleva la dependencia de un solo fabricante. ICa soluciona este problema mediante una interfase de interoperabilidad común acordada por más de 500 empresas de todo el mundo. Dentro del segundo capítulo también se describen los tipos de sistemas de control en los que la utilización de ICa puede aportar beneficios. En un principio, la idea fundamental consistía en resolver el problema de la integración en grandes sistemas de control. Tras las primeras versiones del gestor de objetos se observó la posibilidad de extender ICa a sistemas de control distribuido con otras características. Fundamentalmente, sistemas distribuidos y empotrados con características de tiempo real ya fueran críticas o no. De esta manera, existen diferentes versiones del ICa ORB que pueden embeberse bien en grandes sistemas o en sistemas empotrados con características de tiempo real muy diferentes, manteniendo la gran ventaja de la interoperabilidad entre todas ellas. Esto permite obtener una verdadera integración total, tanto vertical como horizontal en todas las capas de la jerarquía de control. En ICa se emplea el término Total Integration para referirse a esta estrategia. La estructura organizativa que más ha penetrado en el diseño de los sistemas de control es la arquitectura en capas. En el caso del control distribuido es una consecuencia natural de los diferentes tipos de tareas que se deben realizar en el sistema. La estructura en capas permite organizar lógicamente desde los lazos básicos de control hasta planificación de la producción de la planta desde el punto de vista administrativo o de la gestión de los proveedores. Las capas del sistema de control son un medio para reducir la complejidad del mismo. Las funciones similares se agrupan en las mismas capas y se desarrollan independientemente del resto. La comunicación se limita a las capas adyacentes con lo que se reduce el acoplamiento entre las partes del sistema, aumentando al mismo tiempo la reusabilidad del software. Las capas ofrecen una interfaz de servicios que es utilizada por las capas superiores para realizar las tareas de control. A su vez, en los sistemas de control, las capas inferiores proporcionan información a las superiores sobre el estado del proceso. Este mecanismo de comunicación puede ser implementado, por ejemplo, en la forma de un gestor de eventos. En general, la estructura en capas corresponde a un punto de vista del sistema de control en el que simplemente se ha realizado una división lógica de la funcionalidad del mismo. Existe otro punto de vista arquitectónico del sistema en el que el énfasis se encuentra en la implementación de cada una de las capas que componen el sistema de control. Así que en realidad, la arquitectura software de un sistema de control complejo se puede definir como híbrida. Híbrida en el contexto de la arquitectura software significa que existe en el sistema una mezcla de estilos arquitectónicos que pueden ser observados mediante la contemplación de una parte u otra del sistema. La idea es que desde un punto de vista la arquitectura ayuda a gestionar la complejidad del sistema de control mientras que desde otro u otros puntos de vista la arquitectura permite controlar la complejidad del desarrollo del sistema software. También se presenta información al lector sobre los elementos básicos de los sistemas distribuidos de control. En el mundo real las cosas son más complejas que la simple cadena sensor, control, actuador y planta. Incluso en ese bucle sencillo existen elementos especializados dependiendo del área de la industria para la que se diseñe el sistema de control. El efecto de las comunicaciones en el sistema de control no es despreciable y la consideración de latencias, retrasos de transmisión o jitter entre otros introducen nuevos problemas en el diseño del control del proceso. Al mismo tiempo, el sistema de control incorpora interfases hombre-máquina y software de tipo SCADA para facilitar a los operadores el seguimiento y supervisión del estado de la planta. En las capas superiores de la jerarquía de control es posible encontrar desde sistemas de ejecución del proceso de fabricación (MES), que controlan todos los procesos de fabricación en tiempo real, hasta ERPs que realizan una tarea parecida a la del MES sólo que su alcance temporal, al contrario que en el MES, está fijado en el medio y largo plazo. Como ejemplo de los elementos básicos que se pueden encontrar en el entorno del proceso a controlar se detallan varios productos comerciales de utilización en diferentes ámbitos en los que ICa ha demostrado su utilidad. Se hace especial hincapié en los Intelligent Electronic Devices (IED) o dispositivos electrónicos inteligentes como elementos capaces de realizar la función de varios dispositivos en uno solo y como elemento básico sobre el que establecer un sistema de inteligencia distribuida. Este tipo de elementos es la base de los grandes sistemas distribuidos de control. Como contrapunto, también se ofrecen ejemplos de dispositivos que actúan en sistemas distribuidos y empotrados con características de tiempo real crítico. Por ejemplo, se muestra el hardware de un elemento para aplicaciones drive-by-wire que utiliza la tecnología de comunicaciones TTP soportada por ICa. Como se ha mencionado, el mecanismo de comunicaciones empleado en los sistemas distribuidos de control tiene gran importancia al no ser despreciables los retardos inherentes a la red. Dependiendo del tipo de sistema de control distribuido, existirán diferentes tipos de redes (incluso dentro del mismo sistema) con características temporales diferentes. En general, cuanto más cerca del proceso a controlar se encuentre la red, más exigentes serán los requisitos temporales y de otros tipos (por ejemplo, de tolerancia a fallos). En el ámbito industrial, estas redes encajan aproximadamente en las distintas capas de la arquitectura del sistema de control distribuido. Se puede distinguir entre las que se ha venido a denominar red de campo o fieldbus, red de control y la red de gestión o empresarial. Cada una juega su papel dentro del sistema de control y se incluye información sobre los protocolos dominantes en el mercado para cada una de ellas. A su vez, también se considera el caso de los pequeños sistemas distribuidos (por ejemplo, la red de un automóvil) y los protocolos disponibles en el ámbito de dicho tipo de sistemas. La tecnología de los sistemas distribuidos de control se encuentra en constante evolución y ese hecho afecta a la topología de las redes de control distribuido. El segundo capítulo incluye información sobre las tendencias pasadas, presentes y futuras en lo que a topología y distribución de la inteligencia en el sistema de control se refiere. En la parte final del capítulo se presentan algunos sistemas que constituyen el estado del arte en cuanto a control distribuido hoy en día, y que se encuentran disponibles de manera comercial. Dado el interés de esta Tesis en un amplio rango de sistemas distribuidos diferentes, se muestran tres sistemas en los que ICa puede ser objeto de aplicación. El primer sistema se denomina Experion Process Knowledge System de Honeywell. Es un sistema para grandes plantas que abarca toda la jerarquía de capas de control. El segundo es un sistema distribuido, empotrado y crítico en cuanto a tiempo real y en cuanto a seguridad (safety, no security). Se trata del control de presión de cabina del mayor avión comercial construido hasta la fecha, el Airbus 380. El tercer ejemplo es diferente de los dos primeros. Se trata del sistema de control distribuido de las subestaciones de alimentación eléctrica de los trenes de Alta Velocidad Española que hacen el recorrido Madrid-Barcelona. Uno de los objetivos primordiales de este sistema de control es el de la alta disponibilidad del mismo en un entorno geográficamente muy distribuido. El capítulo finaliza con una crítica de las principales limitaciones de cada uno de los sistemas expuestos. En el primer ejemplo, el sistema de Honeywell, el principal problema es el de la dependencia de un fabricante o proveedor que proporciona el conjunto del sistema. En el segundo ejemplo, el problema es la falta de herramientas adecuadas para el desarrollo del sistema de control distribuido. Cualquiera que haya trabajado en sistemas empotrados conoce las limitaciones de las herramientas de desarrollo. Añádase a esto la dificultad de hacer el sistema distribuido. Finalmente, en el caso del sistema de control del AVE, el problema proviene de la cantidad de proveedores diferentes que intervienen en el sistema de control, de los diferentes protocolos de comunicación empleados y de la dificultad de que el conjunto funcione con la fiabilidad requerida. ICa proporciona una solución a estos problemas mediante un marco común de integración que es útil en sistemas de control distribuido de cualquier tamaño. El tercer capítulo, Middleware de Componentes Distribuidos, discute los diferentes modelos de middleware existentes en la actualidad. De una manera sencilla podemos entender el middleware como el lazo de unión entre todos los elementos de un sistema que les ayuda a interactuar. El middleware media entra las aplicaciones y la red y el sistema operativo, aislando a la aplicación de todos los detalles de bajo nivel en la construcción del sistema. Para conseguir ese aislamiento el middleware realiza una abstracción para la programación del sistema distribuido con la que todos los módulos del mismo deben cumplir. El middleware proporciona a la aplicación una interfaz de programación que permite utilizar los diferentes servicios proporcionados por el mismo. Algunos de esos servicios son los siguientes: • Localización de servicios en la red de una manera transparente. Esto significa que la distribución del sistema se realiza de una manera transparente al mismo. Para los componentes del sistema es como si éste fuera monolítico. • • • • La aplicación es independiente de los servicios de red. Las aplicaciones no son conscientes de la red. El middleware se encarga de la gestión de los detalles de acceso a la misma. El middleware proporciona fiabilidad y disponibilidad. La programación a nivel de red es una tarea compleja. El middleware proporciona una manera probada en miles de aplicaciones de pegar los componentes de una aplicación distribuida. El middleware es capaz de ser escalado en capacidad sin pérdida de funcionalidad. Proporciona servicios que permiten controlar el ciclo de vida de los componentes del sistema distribuido y controlar los recursos disponibles para el conjunto del sistema. Resuelve el problema de la heterogeneidad, permitiendo la construcción de sistemas donde existe heterogeneidad de hardware, software, sistemas operativos, protocolos de red y lenguajes de programación. Se puede realizar una clasificación taxonómica del middleware de acuerdo con las formas siguientes. • • • • Monitores de transacciones (TPM). Middleware de llamadas a procedimientos remotos (RPC). Middleware orientado a mensajes (MOM). Middleware orientado a objetos (OOM u ORB). La clasificación distingue los tipos de middleware de acuerdo únicamente a los diferentes mecanismos utilizados en la comunicación entre las partes que componen el sistema distribuido. En esencia el criterio se basa en que todos los tipos de middleware definen un esquema uniforme de acceso al resto de los componentes del sistema distribuido. El TPM es un middleware de tipo cliente / servidor específicamente diseñado para aplicaciones transaccionales. Es una tecnología madura, que se utiliza fundamentalmente en aplicaciones que gestionan datos, bases de datos, emisión de órdenes (por ejemplo de compra), etc. Este tipo de middleware se diseña para proporcionar acceso a miles de clientes en un entorno distribuido cliente / servidor mediante el control y asignación de la peticiones de los clientes a una serie de rutinas de servicio que multiplexan las transacciones. Las rutinas de servicio son críticas en esta arquitectura y son las únicas que interactúan con el sistema servidor evitando a éste tener que realizar la gestión de miles de peticiones simultáneas. La ventaja es que el sistema servidor sólo ve un pequeño número de rutinas con las que intercambia información y no los miles de clientes que en realidad existen. El acceso de los clientes se ordena en función de la prioridad asignada al servicio requerido. Otra ventaja es que la lógica de la aplicación puede moverse de los clientes al monitor de transacciones resultando mucho más sencillas las tareas de administración y actualización de los sistemas. La tecnología TPM es fácilmente escalable cuando el número de clientes crece, ya que siempre es posible añadir más monitores de transacciones para gestionar mejor la multiplexación de las peticiones de los clientes. Además, la mayoría de los productos comerciales proporcionan facilidades para la gestión del equilibrio de carga o características avanzadas de gestión y tolerancia a fallos. El mercado se encuentra dominado por unos pocos fabricantes entre los que se encuentran BEA, IBM o HP. El middleware tipo RPC es una infraestructura tipo cliente / servidor que permite a los desarrolladores acceder a los componentes de un sistema distribuido mediante llamadas a funciones remotas. En este caso, no existe una capa de software que implementa el middleware. El middleware se encuentra embebido en las aplicaciones cliente y servidor mediante la inclusión de ficheros stub que se compilan y enlazan con el resto de la aplicación. Los stubs son invocados por el programa cliente cuando se intenta llamar a una función que se encuentra en el servidor RPC remoto. Este mecanismo actúa de forma síncrona aunque existen algunos fabricantes que ofrecen funcionalidad para realizar llamadas asíncronas. La ventaja del mecanismo RPC es que enmascara las llamadas a funciones remotas como si se estuvieran realizando de una manera local. Como las llamadas se realizan de manera síncrona, en aplicaciones de cierta complejidad es necesario recurrir a la programación concurrente lo que sin duda hace más complicado el desarrollo de los sistemas. La mayor crítica o desventaja de los sistemas basados en RPC es que proporcionan un modelo de programación de relativo bajo nivel enfocado hacia procedimientos. Existen multitud de situaciones donde la tecnología de objetos distribuidos proporciona mejores soluciones. El mercado de soluciones RPC se nutre de dos o tres fabricantes como Entegrity Solutions, Sun o HP. El middleware orientado a mensajes o MOM se basa en una arquitectura tipo cliente / servidor que soporta invocaciones asíncronas entre clientes y servidores. En un sistema basado en MOM los mensajes no se envían directamente entre cliente y servidor. Existe una cola de mensajes tanto en el lado del cliente como en el del servidor. El propósito de la cola de mensajes es su almacenamiento en el caso de que la aplicación que debe procesar los mensajes de la cola se encuentre ocupada realizando otras tareas. En esta arquitectura, una vez que el emisor ha enviado el mensaje puede continuar su ejecución. No queda bloqueado esperando respuesta. El mensaje queda almacenado en la cola de mensajes que intentará enviarlo al programa receptor hasta que éste lo acepte. Una vez que el receptor construya el mensaje de respuesta será enviado a su cola de mensajes hasta que el emisor que inició la comunicación sea capaz de aceptarlo. La ventaja de los sistemas basados en MOM es que desacopla los procesos cliente y servidor. Esta era una de las mayores desventajas del mecanismo RPC. Como MOM permite la interoperación de módulos que son independientes entre sí (se puede enviar un mensaje a un servidor que no se encuentra operativo en un momento dado) es una arquitectura bastante apreciada en sistemas de integración empresarial. El mercado se encuentra controlado por grandes empresas que desarrollan su actividad en el mundo empresarial. IBM o Software AG se encuentran entre los principales proveedores de sistemas MOM. El último tipo de middleware es el orientado a objetos y componentes. Este tipo de middleware hace especial hincapié en el concepto de interfaz en un entorno de programación orientado a objetos. Todos los servicios son accedidos a través de una interfaz y no es necesario conocer ningún detalle de implementación de los mismos. El middleware orientado a objetos está especialmente pensado para soportar el modelo de programación orientado a objetos obteniendo las mismas ventajas que tienen estos sistemas. De esta manera, mediante el middleware orientado a objetos se pueden construir sistemas que representan bien sistemas del mundo real que son distribuidos por naturaleza. El middleware orientado a objetos utiliza un manager orientado a objetos (OOM) o un broker de peticiones de objetos (ORB) como la capa software que media entre los objetos local y remoto. Los objetos remotos exponen su interfaz que es accedida de manera local mediante objetos proxy o representantes de los objetos remotos. El proxy envía las peticiones al ORB que toma las acciones necesarias para transmitir la petición del servicio al objeto remoto. Entre las ventajas que aporta este tipo de middleware se pueden citar las siguientes: • • • • • • • No existen clientes ni servidores, sólo objetos que ofrecen servicios y otros que reciben servicios. Transparencia en cuanto a la situación del objeto. La localización del proveedor del servicio es transparente al usuario del servicio. La distribución de los servicios se puede realizar en cualquier momento. Los objetos proveedores de servicios se pueden migrar dinámicamente sin afectar a los usuarios de los mismos. El middleware introduce el concepto de servicios generales como el servicio de nombres, de suscripción a eventos o el de gestión del ciclo de vida de los objetos. La arquitectura del sistema se hace abierta y escalable. El middleware hace al sistema independiente de los lenguajes de programación y del hardware utilizado. En las actualidad, las tres implementaciones de este tipo de tecnología dominantes en el mercado son las basadas en la especificación de CORBA del OMG, la tecnología Java / RMI de Sun MicroSystems y la tecnología DCOM de Microsoft. En este capítulo se abunda en la manera que cada una de estas tecnologías construye su visión del middleware orientado a objetos y en una comparación de la complejidad relativa de cada una de ellas. El cuarto capítulo se encuentra dedicado a los sistemas basados en la tecnología CORBA. Introduce los conceptos básicos de la tecnología y las especificaciones más relevantes para el mundo del control distribuido desarrolladas por el OMG. El OMG es una organización sin ánimo de lucro fundada en 1989 con el objeto de desarrollar especificaciones independientes de los fabricantes que permitan fomentar la utilización de la tecnología de objetos distribuidos. La organización cuenta hoy en día con más de 500 miembros y mantiene contacto permanente con otras organizaciones de estandarización como ISO o W3C. La visión de los sistemas de objetos distribuidos del OMG está basada en la arquitectura de gestión de objetos u OMA. La OMA especifica cómo construir sistemas abiertos de objetos distribuidos mediante el uso de un broker de objetos (ORB) y una colección de servicios predefinidos. La arquitectura se basa en cuatro componentes que pertenecen a dos tipos en función del servicio que proporcionan: ORBs y servicios a objetos, y objetos de aplicación y utilidades genéricas. Estos elementos describen los bloques principales que componen la OMA. • • • • • Gestor o broker de peticiones de objetos (ORB): Es el elemento central de la arquitectura que permite la transmisión de las peticiones y sus respuestas entre los objetos CORBA. Servicios a objetos (CORBAservices): Proporcionan servicios genéricos de bajo nivel. Son útiles en la práctica totalidad de aplicaciones. Ejemplos de servicios genéricos son el servicio de nombres o el servicio de notificación. Utilidades horizontales (CORBAfacilities): Son servicios genéricos de alto nivel por lo que no pertenecen a la categoría de servicios. Ejemplos son utilidades de impresión, de correo electrónico o de gestión del tiempo. Utilidades verticales (Domain CORBAfacilities): Proporcionan servicios estándar en un campo específico de aplicación (por ejemplo en servicios sanitarios, telecomunicaciones o transporte). Objetos de aplicación: Proporcionan un servicio específico dentro de un sistema dado. Son servicios desarrollados para construir un sistema particular. Como todas las arquitecturas software, la OMA resulta de un compromiso resultante de dirimir las tensiones existentes entre los requisitos de la misma: facilidad de construcción con la tecnología existente, generalidad que permita su construcción por muchos fabricantes diferentes, transparencia entre aplicaciones cliente y objetos proveedores de servicios, interoperabilidad entre implementaciones de la arquitectura y construcción de puentes hacia otras tecnologías, capacidad de evolución, extensibilidad, etc. Los servicios CORBA son interesantes en la práctica totalidad de aplicaciones distribuidas porque proporcionan funcionalidad preconstruida que se puede reutilizar. Son útiles de manera horizontal a todos los dominios de aplicación. Algunos de los servicios más interesantes son los siguientes. • • • • • • • Tiempo extendido. Es una especificación que extiende el servicio de tiempo para que sea capaz de representar relojes con diversas características (resolución, precisión, estabilidad, etc.). Es interesante cuando se utiliza más de un reloj en un sistema. Servicio de eventos. Permite desacoplar los objetos que solicitan servicios de aquellos que los proporcionan. Permite la comunicación asíncrona de eventos mediante peticiones CORBA. Servicio de ciclo de vida. Proporciona una interfaz para crear, eliminar, copiar y mover objetos. Permite a los solicitantes del servicio realizar operaciones de ciclo de vida sobre los objetos que residen en diferentes localizaciones. Servicio ligero de registro. Es un servicio que permite la gestión de registros en aplicaciones de tiempo real o empotradas. Servicio de nombres. Permite designar a los objetos mediante una notación comprensible por las personas. Asocia referencias interoperables a objetos con nombres identificables por las personas. Servicio de exteriorización. Proporciona un protocolo que permite exteriorizar e interiorizar objetos. Exteriorizar es registrar el estado de un objeto en una corriente de datos. Interiorizar es el proceso contrario. Servicio de estado persistente. Proporciona una interfaz para la retención del estado de los objetos. Permite restaurar el estado dinámico de un objeto a partir de su estado persistente. Este servicio se basa en el servicio de exteriorización. De las especificaciones de CORBA, la más interesante para los sistemas de control distribuido es la especificación de CORBA para tiempo real. Esta especificación extiende CORBA de manera que sea posible controlar los recursos del sistema y así aumentar la predecibilidad de una aplicación de objetos distribuidos. Existen dos especificaciones de tiempo real en CORBA. La primera contempla los sistemas basados en un esquema de prioridades fijas mientras que la segunda soporta la planificación dinámica del sistema de objetos distribuidos. Dada la complejidad de la planificación dinámica de un sistema de tiempo real, los ORBs que existen en la actualidad sólo soportan el mecanismo de planificación basado en prioridades fijas (salvo alguna excepción experimental). El objetivo de la especificación de CORBA para tiempo real con prioridades fijas es el de permitir la construcción de sistemas distribuidos que sean predecibles. En la especificación definida por el OMG, esto quiere decir que a) las prioridades de los hilos de ejecución se deben respetar entre clientes y servidores, b) se debe acotar la inversión de prioridad durante el procesamiento de invocaciones entre objetos y c) se deben acotar las latencias de las invocaciones a servicios. La especificación se apoya en extensiones de entidades clave de CORBA (ORB, adaptador de objetos, modelo de concurrencia, etc.) y de los servicios proporcionados por ellos. La especificación de tiempo real de CORBA permite gestionar tres tipos de recursos en un sistema distribuido. • • • Recursos de procesador. Recursos de memoria. Recursos de comunicaciones. Respecto a la gestión de los recursos utilizados por el procesador, mediante la interfaz proporcionada por la especificación de CORBA para tiempo real es posible utilizar mecanismos de gestión para determinar la prioridad con la que se deben ejecutar las invocaciones CORBA. De la misma manera, los objetos servidores pueden controlar el número de hilos de ejecución que se utilizarán para resolver peticiones concurrentes y simultáneamente limitar la prioridad de los mismos. Adicionalmente, el modelo de control de recursos define un elemento de sincronización que utiliza la misma semántica que la empleada por los hilos de ejecución para minimizar las inversiones de prioridad en el sistema. En cuanto a los mecanismos de la gestión de la prioridad se debe tener en cuenta que los sistemas que deseamos diseñar son heterogéneos y distribuidos. Por este motivo CORBA define el concepto de prioridad. Es posible entonces, diseñar el sistema utilizando prioridades CORBA que a su vez serán relacionadas a través del middleware con las correspondientes prioridades nativas de cada sistema operativo en tiempo de ejecución. Para coordinar las prioridades dentro del sistema, la especificación define dos modelos de prioridad. • • Modelo de prioridad declarada por el servidor. En este modelo la prioridad que utiliza el servidor para atender una invocación viene definida por el mismo. La prioridad que se utilizará para atender la petición de un cliente se encapsula en las referencias a objetos, permitiendo de esta forma que los clientes conozcan esta información. Modelo de prioridad declarada por el cliente. Existen ocasiones en las que es mejor mantener en el lado del servidor la prioridad de ejecución que el cliente utiliza. Es una manera de evitar las inversiones de prioridad en el camino de ejecución de la petición. En este modelo, cada invocación o petición al servidor encapsula la prioridad a utilizar siendo posible que cualquier hilo de ejecución que se utilice en el lado del servidor se ejecute con la prioridad deseada por el cliente. La prioridad encapsulada es una prioridad CORBA de manera que mediante el mapping de prioridad definido por la especificación pueda ser transformada en una prioridad nativa de cada sistema operativo utilizado. En los casos anteriores, el control más fino que se puede obtener respecto a la prioridad de ejecución es a nivel de objeto pero pueden existir situaciones donde lo que se necesita es controlar la prioridad a la que se va a ejecutar una determinada invocación. Para este caso, la especificación define las llamadas transformaciones de prioridad que permiten cambiar dinámicamente la prioridad a la que se ejecuta una invocación. Las transformaciones de prioridad son de dos tipos, inbound y outbound. La primera se ejecuta cuando el middleware del lado servidor recibe una invocación y antes de que la misma sea despachada al objeto que soporta la operación solicitada. Las transformaciones outbound ocurren cuando un objeto servidor se encuentra procesando una operación y tiene a su vez que invocar a otro objeto servidor. En ese momento se produce una transformación de tipo outbound. Otro elemento que se puede controlar en el lado del servidor son los hilos de ejecución empleados para atender las invocaciones de las aplicaciones cliente. Para este propósito se utiliza el concepto de threadpool o pila de hilos de ejecución. Un threadpool es una colección de hilos de ejecución que se puede asociar a los adaptadores de objetos de un servidor CORBA. Mediante la configuración de las características del threadpool es posible diseñar un servidor concurrente con muy poco esfuerzo ya que el middleware se encarga de la gestión de todos los hilos de ejecución. En la especificación de CORBA para tiempo real existen dos modelos de threadpool, con y sin carriles. En el modelo sin carriles simplemente se crea una colección de threads con una serie de propiedades comunes a todos ellos (por ejemplo, prioridad de ejecución). Los threads se crean estáticamente, lo que permite que estén preparados en cualquier momento para atender una invocación por parte de un cliente. También es posible especificar el número de threads que el threadpool puede crear dinámicamente en caso de que sea necesario (para manejar situaciones en las que se producen incrementos puntuales de uso del servidor). En el caso de que el threadpool permita la utilización de carriles, se pueden asignar grupos de thread a cada carril y asignar a cada carril una prioridad diferente. De esta manera es posible realizar una partición de la prioridad en función del uso que se le vaya a dar en el sistema a desarrollar. Cada carril permite también controlar el número de threads estáticos que contiene y el número de threads dinámicos que se pueden crear. Una última característica de este tipo de threadpool es que se permite el préstamo de threads entre carrilles. Los carriles de mayor prioridad pueden promocionar threads de carriles de menor prioridad a una prioridad superior. Una vez que el thread promocionado ha atendido la petición para la que se promocionó es devuelto a su carril original. Para que las aplicaciones construidas sobre el modelo de concurrencia del middleware sean consistentes es necesario que exista un modelo de sincronización acorde al de concurrencia. Para esto y como recurso de control del procesador existe una interfaz para acceder a los objetos mutex que utiliza el ORB. Se asegura así que el mecanismo de sincronización utilizado por el ORB y la aplicación que se ejecuta sobre él son del mismo tipo. El segundo tipo de recurso que se puede controlar es el relativo a la memoria. De cara a gestionar la misma se puede indicar en el lado del servidor, en el momento de creación de los threadpool, el buffer de memoria que se desea utilizar para almacenar las invocaciones que debe atender el threadpool cuando todos sus hilos de ejecución se encuentran ocupados atendiendo otras invocaciones. Además, se puede configurar el tamaño del stack que utilizará cada thread para atender las invocaciones de las aplicaciones cliente. El tercer tipo de recurso que se puede gestionar son los relacionados con los enlaces de comunicaciones. Controlando los recursos utilizados para las comunicaciones es posible configurar los parámetros o propiedades de un determinado protocolo. En realidad, se pueden configurar dos protocolos diferentes, el utilizado por el ORB (GIOP en el caso habitual) y el correspondiente al transporte de red (TCP / IP por defecto). La interfaz permite asignar varios protocolos a un mismo ORB y el orden de asignación indica el orden de preferencia en el que se usarán los protocolos para establecer comunicación. La selección de los protocolos en CORBA se realiza aplicando políticas de control de la calidad del servicio. Estas políticas se pueden aplicar tanto en el lado del cliente como del servidor. Cuando la política se aplica en el lado del servidor la información que contiene se publica en las referencias interoperables de dicho servidor de tal forma que los clientes son capaces de conocer los protocolos que soporta el servidor. Cuando la política se aplica en el lado del cliente (la política se puede asignar en la parte del servidor y se exportará al cliente o puede ser aplicada directamente en el cliente) permite configurar a éste para seleccionar un protocolo determinado. Además, en el lado del servidor la política se aplica a nivel del objeto mientras que en el cliente es posible cambiar dinámicamente en cada invocación el tipo de protocolo a utilizar. Además de realizar una selección de protocolos, hay otros factores que intervienen en la predecibilidad extremo a extremo de una aplicación distribuida. Un primer factor consiste en la determinación del momento en el que se establece la conexión entre el cliente y el servidor. En el caso más habitual, se establece la conexión en el momento de realizar la primera invocación por lo que está petición sufre el retardo relacionado con el proceso de establecimiento de una conexión. En un ORB diseñado para tiempo real este retardo se puede eliminar estableciendo una conexión de manera explícita entre el cliente y el servidor antes de la primera invocación. En el caso general, un ORB no permite controlar el número de conexiones que se establecen entre un cliente y un servidor y compartirá una o más conexiones entre los clientes de un servidor. Esta filosofía no es razonable en aplicaciones de tiempo real por lo que la especificación define políticas para la utilización de conexiones privadas, multiplexadas o para establecer conexiones que representan bandas de prioridad. El ORB es responsable en este caso de la transmisión de cada invocación por la conexión que representa a la banda que contiene la prioridad deseada para la invocación. Mediante las interfases expuestas de control de recursos del procesador, memoria y comunicaciones se puede conseguir mejorar la predecibilidad de un sistema basado en CORBA. Existe una segunda especificación de CORBA para tiempo real que ataca el problema de la planificación dinámica aunque no existen implementaciones comerciales debido a la complejidad que representa la construcción de un ORB de este tipo. La especificación se basa en el concepto de hilo de ejecución distribuible. Este es un hilo de ejecución que puede traspasar los límites físicos de una máquina determinada de manera que se conserven las propiedades del mismo a lo largo del camino de ejecución. El capítulo 4 contiene información más detallada sobre esta especificación. Desafortunadamente, la especificación de CORBA para tiempo real se basa en la especificación de CORBA. Esto quiere decir que el tamaño del ORB de tiempo real es mayor que el de un simple ORB de CORBA. En sistemas, habituales en control, donde existen restricciones de memoria, recursos de procesador, etc. esta situación es claramente prohibitiva. La especificación minimum CORBA proporciona un perfil de CORBA reducido para este tipo de aplicaciones manteniendo al mismo tiempo la compatibilidad con la especificación general de CORBA. Esta Tesis tiene en cuenta estos problemas y presenta un middleware de tiempo real en dos versiones; una soporta la especificación de CORBA y otra presenta la interfaz de tiempo real sobre la especificación de minimum CORBA. Es posible entonces tener las ventajas de un broker CORBA de tiempo real sobre la especificación para sistemas empotrados. El quinto capítulo desarrolla una de las partes más importantes de esta Tesis y muestra las características del broker CORBA desarrollado para aplicaciones de control distribuido. El capítulo presenta al lector las debilidades de CORBA a medida que su aplicación se realiza más cerca del proceso a controlar ya que los requisitos del sistema de control son cada vez más exigentes. El broker CORBA para tiempo real de ICa (Arquitectura de Control Integrado) es el resultado de un esfuerzo a largo plazo para obtener una infraestructura que permita desarrollar sistemas de control distribuido de una manera sencilla. Dado el interés de esta tecnología de cara a los objetivos sociales y económicos de la Unión Europea, esta iniciativa ha sido extensamente financiada en el marco de los programas ESPRIT e IST. ICa surgió a partir de la necesidad de reducir el esfuerzo de programación a nivel de comunicaciones en aplicaciones distribuidas. El primer intento de resolver el problema, hacia el final de los años 80, dio lugar a la librería CCL (Complex Communications Layer) que encapsulaba el API de programación de la red a nivel de TCP/IP y que fue desarrollada durante el proyecto CONEX. Fue también hacia el final de 80 y principios de los 90 cuando se fundó el OMG y cuando comenzó a tener gran relevancia la Arquitectura de Gestión de Objetos u OMA. Nuestro grupo de investigación UPM-ASLab se dio cuenta de la importancia que la arquitectura tendría en el futuro y decidió construir una capa software sobre la librería CCL que le permitía seguir la filosofía de desarrollo orientada a objetos. Este primer ORB ya se denominó ICa pero no era un broker CORBA al no cumplir con la especificación. Al mismo tiempo, siendo un gran avance en nuestra línea de investigación incluyendo aspectos como la configuración del marshalling o proporcionando timeouts programables para las invocaciones, no consideraba aspectos fundamentales para los sistemas de tiempo real y de control distribuido tales como la optimización de los algoritmos del ORB o la configuración de la calidad de servicio en el sistema. También se desarrolló un compilador denominado ADL que utilizaba un lenguaje de definición de agentes para construir los stubs de comunicaciones que las aplicaciones necesitaban. Muchas de las circunstancias que impedían un uso óptimo del ORB en el mundo del control fueron identificadas y con la financiación obtenida en el proyecto DOTS del programa europeo IST se construyó un ICa ORB completamente nuevo que se utilizó con éxito en el área de sistemas de automatización de subestaciones eléctricas. En este punto ICa ORB ya era un broker CORBA para tiempo real especialmente preparado para su uso en sistemas industriales. Aún así, los sistemas que se podían atacar eran aquellos de tiempo real no crítico y el objetivo que se quería alcanzar era el de poder utilizar ICa en un abanico de sistemas tan amplio como fuera posible. Con este objetivo en mente se consiguió más financiación y se lanzó el proyecto HRTC mediante el que se ha dotado a ICa con la capacidad de trabajar en entornos de tiempo real crítico. En el caso de ICa se han estudiado las causas de falta de determinismo en el ORB de manera que se aumente la predecibilidad de ICa ORB. Algunas de las mejoras del broker son las siguientes. • • • El código generado automáticamente por el compilador de IDL para los esqueletos en el lado del servidor y para los stubs en el lado del cliente se ha mejorado notablemente de manera que el paso de información de las invocaciones a través de estos adaptadores entre el middleware y el código de la aplicación sea muy eficiente. Lo mismo ocurre con el encapsulado de las invocaciones a formato GIOP. ICa ORB utiliza GIOP a nivel de protocolo de mensajes del ORB para garantizar la interoperabilidad con otros ORBs comerciales. Para que un broker sea predecible la gestión de la memoria debe ser controlada. El uso continuado de memoria dinámica (ciclos de solicitud y liberación de memoria) es costoso en términos de recursos y puede llevar a situaciones de fragmentación de la misma. ICa ORB permite la configuración de los buffers utilizados para el almacenamiento de las invocaciones de manera que no exista una gestión dinámica de esta memoria. El transporte por defecto en CORBA es IIOP. Esta transporte esta basado en TCP/IP que no es predecible. Este problema se ha solventado mediante la • • incorporación de un mecanismo de plug-in de transportes en ICa ORB que permite la utilización de transportes predecibles. Despacho de hilos de ejecución. Otras de las partes importantes para evitar la pérdida de predecibilidad consiste en evitar la inversión de prioridad al asignar hilos de ejecución para servir invocaciones en el lado del servidor. ICa ORB incorpora un mecanismo reactivo que permite asignar hilos de ejecución a invocaciones cuando se detecta actividad en los buffers del plug-in del transporte de red utilizado. Despacho de invocaciones. La identificación del objeto servidor que implementa la operación solicitada debe realizarse a la mayor brevedad, sin incurrir en retrasos innecesarios. Para este propósito ICa ORB incorpora mecanismos de búsqueda basados en algoritmos O(1) que proporcionan una duración de la búsqueda constante en el tiempo. Las optimizaciones anteriores son de utilidad en cualquier sistema de tiempo real, crítico o no. Existen otras mejoras que ICa ORB incluye específicamente para sistemas críticos. • • • • En el modelo general de CORBA la interacción entre los objetos distribuidos se realiza de un modo síncrono. Incluso las invocaciones oneway incorporan un mecanismo de control de errores que las convierte en bidireccionales y síncronas. A pesar de ser este mecanismo el mejor de los posibles en cuanto a la fiabilidad obtenida, la mayoría de las aplicaciones de control (sobre todo las empotradas) utiliza mecanismos de comunicación no fiable, unidireccional y no orientado a la conexión. Es posible utilizar llamadas oneway puras en ICa ORB imitando de esta manera el mecanismo de comunicación utilizado en muchos sistemas de tiempo real. ICa puede ser utilizado en sistemas disparados por eventos o en sistemas disparados por tiempo. Las arquitecturas disparadas por tiempo son más frecuentes en sistemas de misión crítica y compensan su falta de flexibilidad exhibiendo un comportamiento más determinista que los sistemas dirigidos por eventos. ICa ORB permite obtener información temporal sobre el progreso del sistema. Para ello define una arquitectura en la que el transporte de comunicaciones es el encargado de mantener el tiempo del sistema. La gestión del tiempo se ha incorporado a la interfaz del marco extensible desarrollado para la utilización de plug-ins de transportes. A su vez, se ha extendido el modelo de objeto CORBA para acceder a la interfaz del transporte y permitir la obtención de información relacionada con el progreso temporal de las invocaciones realizadas en el sistema. Como trabajo futuro, se propone un modelo de lenguaje de definición de interfases (IDL) en el que es posible definir las tareas periódicas, aperiódicas y esporádicas. • • ICa ORB proporciona un servicio de registro del tiempo en el que las invocaciones se completan mediante políticas aplicables a nivel de objeto. Finalmente, existen estadísticas sobre los protocolos de transporte utilizados que permiten conocer información en tiempo de ejecución tales como retardos máximo, mínimo, medio, etc. Para demostrar la idoneidad de las mejoras realizadas, durante el desarrollo del proyecto HRTC se han construido dos transportes de tiempo real crítico. El primero, basado en RT Ethernet, puede ser utilizado en sistemas disparados por eventos mientras que el segundo, que utiliza el protocolo TTP, está orientado a sistemas disparados por tiempo. Ambos transportes han sido utilizados junto con ICa en aplicaciones empotradas como se muestra en capítulo dedicado a las aplicaciones de ICa. El sexto capítulo, presenta una metodología para el desarrollo de sistemas de control basados en ICa (ICm). El objetivo final de ICa es permitir a los desarrolladores construir los sistemas de control que imaginen. Para ello, no es suficiente con disponer de la infraestructura software que ICa ORB proporciona. Es también necesario disponer de una metodología que ayude a conseguir el comportamiento y características de tiempo real adecuadas para los sistemas distribuidos de control desarrollados con ICa. La metodología ICm contiene las líneas maestras del proceso a seguir para construir sistemas distribuidos siguiendo prácticas de ingeniería. La metodología propuesta recomienda un proceso de desarrollo en el que se siguen técnicas de ingeniería de dominio para la creación sistemática de los modelos de dominio y arquitecturas software de los sistemas de control CORBA. La metodología se basa en la búsqueda de modelos de referencia que permitan realizar desarrollos genéricos para un conjunto de sistemas distribuidos de control. Mediante la traslación del modelo de referencia al conjunto de sistemas y componentes software que lo implementan se consigue la arquitectura de referencia y a partir de la construcción de los activos software que componen el sistema se consigue una arquitectura software para un tipo de sistemas. La idea es obtener líneas de producto para los sistemas de control distribuido que se puedan construir como familias de productos. Los sistemas se desarrollan a partir de los activos software que constituyen el núcleo de la familia de productos. Para que lo anterior sea posible es necesario que la arquitectura software del sistema se desarrolle de una manera estándar. La estandardización es útil porque obliga a elevar el nivel de abstracción y a pensar en las necesidades de múltiples sistemas. Se debe pensar en las necesidades del dominio de aplicación. Esa es la idea básica de la ingeniería de dominio. De esta manera, ICa proporciona una infraestructura básica sobre la que activos software para un determinado dominio de control distribuido pueden ser construidos. En otras palabras, la metodología incide en la utilización de la ingeniería de dominio sobre la infraestructura de ICa para la construcción de activos software que permiten construir conjuntos de sistemas distribuidos de control dentro de un contexto dado. Por ejemplo, sistemas de control de tráfico aéreo o sistemas de control de subestaciones eléctricas. El foco debe situarse en el proceso de ingeniería necesario para desarrollar clases o conjuntos de sistemas distribuidos de control y no soluciones únicas a problemas de control específicos. Al desarrollarse el trabajo en el ámbito de los dominios y no de las aplicaciones es necesario en cada momento decidir el alcance del desarrollo realizado, esto se realiza mediante la técnica de focalización progresiva del dominio. La metodología propuesta se basa en el análisis, diseño e implementación de los elementos del dominio. La utilización de la ingeniería de dominio permite que durante la fase de análisis se acote el dominio de trabajo y que mediante el estudio de sistemas preexistentes se obtenga las características comunes y diferentes de los mismos. Además, el objetivo es obtener una comprensión de las relaciones entre los elementos del dominio. El análisis da como resultado un modelo del dominio que se utiliza para realizar el diseño del dominio teniendo en cuenta la utilización de ICa para el diseño de una arquitectura genérica de la familia de sistemas de control basada en los resultados del análisis y en las clasificaciones existentes de patrones de sistemas y estilos arquitectónicos. La fase de implementación del dominio consiste en la elaboración de repositorios de componentes (este término se usa aquí de forma genérica) reutilizables para el dominio de control. Los componentes se han identificado a partir de la arquitectura desarrollada durante las fases de análisis y diseño del dominio. De esta manera, y mediante la aplicación de la técnica de focalización progresiva del dominio, se construyen diferentes soluciones o activos software (denominados componentes en el párrafo anterior) que se utilizan en la construcción de una familia de sistemas distribuidos de control CORBA. Esos activos se denominan objetos ICa, patrones ICa, activos ICa y subsistemas o sistemas según su grado creciente de complejidad. Un sistema de control concreto se construye a partir de la composición y particularización de los componentes descritos para un problema de control determinado. Para facilitar la utilización de la metodología descrita en el Capítulo 6, cada una de las fases de la ingeniería de dominio para los sistemas de control CORBA proporciona los pasos que se deben seguir en el proceso de desarrollo. La ventaja es que la metodología siempre cuenta con la utilización de ICa ORB como infraestructura básica de manera que gran parte del trabajo a realizar en cada fase ya se encuentra implementado. A lo largo del capítulo, se demuestran los pasos a seguir mediante la presentación de un ejemplo real de la utilización de la metodología consistente en la construcción de la infraestructura necesaria para la construcción de una línea de productos como una familia de productos para la automatización de subestaciones eléctricas. La fase de análisis del dominio de los sistemas de control CORBA consiste en la determinación del alcance e identificación del dominio de control, el análisis sistemático de las características comunes y de la variabilidad de la familia de sistemas del dominio de control. Además, incluye la construcción de un conjunto de requisitos para la familia de sistemas. A pesar de que la metodología da preferencia a la construcción de soluciones de ingeniería de dominio para ICa, es posible tomar en la fase de análisis la perspectiva de la ingeniería de aplicación. El resultado de la aplicación de la metodología desde este punto de vista da lugar a la construcción de soluciones específicas de control distribuido. Es decir, da lugar a la construcción de un sistema de control distribuido. El objetivo del diseño del dominio de los sistemas de control CORBA es tomar los resultados del análisis de dominio realizado con ICm y producir un diseño e implementación genéricos para una familia de sistemas. Para ello es necesario mantener el nivel de abstracción utilizado durante la fase de análisis (en el sentido de que no se piensa en un solo sistema sino en múltiples sistemas). El resultado del diseño es una arquitectura software basada en ICa que es reutilizable en múltiples sistemas. El resultado del proceso de diseño para el dominio proporciona una estrategia de partición del dominio que define los componentes de la arquitectura y la asignación del conjunto de características del dominio a cada uno de ellos. Además, se obtiene un modelo de control que define el modelo de comportamiento de los elementos software que constituyen la arquitectura. La construcción de la infraestructura para la familia de sistemas de control del dominio (de los componentes software de la familia) se apoya sobre ICa, ya que ésta es una infraestructura de infraestructuras o lo que es lo mismo, ICa se ha construido realizando ingeniería de dominio para el dominio de todos los sistemas de control distribuido. La implementación de los sistemas de control CORBA se apoya en las fases previas de la metodología, el análisis y diseño del domino. Estas fases anteriores concluyen con la construcción de una infraestructura que puede ser reutilizada en todos los sistemas del dominio y que permite obtener garantías respecto a los atributos de calidad de los sistemas construidos, sobre todo cuando se compara con lo que se podría obtener al construir los sistemas mediante la simple utilización de ingeniería de aplicación. En función de los resultados obtenidos en las fases previas a la implementación, la construcción de sistemas de control CORBA consiste en gran medida en la selección y configuración de las características adecuadas de la infraestructura desarrollada para el dominio. El objetivo de la metodología es la producción de la menor cantidad de código e implementación ad-hoc como sea posible. En cualquier caso, será necesaria la identificación de las características del sistema no implementadas por la infraestructura del dominio y, o bien su incorporación a la infraestructura básica del dominio, o bien su construcción mediante ingeniería de aplicación. La infraestructura del dominio se refina en un proceso iterativo mediante la construcción de nuevos sistemas de control a partir de la misma. Finalmente, el capítulo relativo a la metodología concluye con algunas valoraciones sobre los resultados obtenidos mediante la metodología propuesta en comparación con los sistemas de producción que se encuentran en uso en el dominio de la automatización de subestaciones eléctricas. Los resultados obtenidos permiten afirmar que el uso de la metodología, del proceso de desarrollo, basado en la arquitectura software y en ICa ORB presenta numerosas ventajas en comparación con el desarrollo típico de una aplicación de control. El séptimo capítulo presenta algunos ejemplos de sistemas de control CORBA desarrollados durante los trabajos de esta Tesis. Uno de los objetivos de la Tesis consistía en la demostración de que una única tecnología podía ser capaz de servir para la construcción de sistemas dispares de control. Desde grandes sistemas distribuidos a pequeños sistemas empotrados con necesidades de comunicación. Este capítulo presenta un resumen de la financiación obtenida para el desarrollo de estos trabajos así como los sistemas construidos que responden a dominios diferentes dentro del dominio más amplio de los sistemas de control distribuido (focalización progresiva del dominio). En general, la financiación de los proyectos de investigación se ha obtenido de la Unión Europea a través de los programas IST y ESPRIT. Los proyectos financiados han sido los siguientes. • • • • ESPRIT Project 22130 - Distributed Information Technology for Strategic Multiobjective Process Control (DIXIT). IST-1999-10251 - Distributed Object Telecontrol Systems and Networks (DOTS). IST-2001-37652 - Hard Real-Time CORBA (HRTC). IST-004669 – COMPonent Approach for Real-time and Embedded (COMPARE ). Este proyecto continua los trabajos de esta Tesis. Cada uno de los proyectos explora diferentes aspectos en el uso de tecnología CORBA. El proyecto DIXIT es el más antiguo y se centra en la interoperabilidad en sistemas heterogéneos para conseguir el control de la planta desde el punto de vista estratégico. Utiliza diferentes técnicas de inteligencia artificial (sistemas fuzzy, redes neuronales, sistemas expertos) para conseguir realizar predicciones sobre el estado de la planta y actuar en consecuencia. El proyecto DOTS se centra en objetivos de telecontrol de sistemas a través de redes. CORBA es el elemento integrador y se estudia la construcción de sistemas a partir de un modelo general del dominio de control. El proyecto HRTC lleva ICa ORB un poco más allá, al acometer su utilización en entornos de tiempo real crítico tanto para grandes sistemas distribuidos como para sistemas empotrados. El resto del capítulo presenta una variedad de sistemas ejemplares construidos con la infraestructura de ICa y su metodología. Existen ejemplos en muchos dominios en los que existen diferentes problemas de control. Una breve descripción de cada uno de ellos es la siguiente. • • • • • • Control Distribuido de Procesos. Es un sistema para realizar el control de una planta química piloto. El objetivo fundamental del sistema es la identificación de los requisitos necesarios en este tipo de sistemas y la realización de experimentos en condiciones de heterogeneidad y de integración de sistemas propietarios. Control Distribuido de un Robot Industrial. El objetivo de este sistema es la demostración de las deficiencias temporales observables en el cálculo del control distribuido mediante la utilización de ICa para tiempo real y de un ORB en el que no se contemple el tiempo real. El objetivo práctico consiste en la captura de manera autónoma de una pelota que una persona lanza al brazo robótico (un manipulador ABB IRB2000). Para ello, se utiliza visión artificial estereoscópica para la determinación de la trayectoria de la pelota en tiempo real. Sistema de Telecontrol de Subestaciones Eléctricas. Otro de los sistemas desarrollados contempla el control distribuido de subestaciones eléctricas. El objetivo es la implementación de servicios estándar para la construcción de este tipo de sistemas y la obtención de interoperabilidad mediante ICa para dispositivos de control de distintos fabricantes. Hydra-Visión. Es un sistema que proporciona vídeo digital a los operadores de la sala de control en una red de área extensa. El sistema se ha utilizado para tener supervisión directa de las acciones de control en plantas hidráulicas de generación de energía eléctrica. Consta de una serie de nodos que generan vídeo y audio digital en formato MPEG y una serie de nodos que consumen el audio y vídeo generados. Las cámaras y micrófonos se pueden controlar de manera remota e incluso se pueden definir blancos para las cámaras o realizar la grabación de cualquier suceso que ocurra en la planta. Un protocolo de interoperabilidad con el SCADA permite el posicionamiento y grabación automática de cualquier evento en la planta controlada. Teleoperación de un Robot Virtual. El objetivo de este proyecto consiste en la demostración de técnicas de objetos distribuidos. Mediante el uso de ICa es posible definir una interfaz genérica y construir un sistema distribuido que permita la utilización de un sistema robótico de cualquier tipo mediante un joystick con realimentación de fuerza, una interfaz gráfica de usuario, una página web o cualquier otra forma de acceso a los servicios del sistema. RiskMan. Es un gestor de riesgos (Risk Manager) para plantas petroquímicas. El sistema realiza la supervisión de los elementos que puedan afectar a la seguridad y al mantenimiento de la planta. Respecto a la seguridad, el objetivo del sistema es la prevención de situaciones anómalas, informando en todo momento al personal adecuado de la planta y lanzando el plan de emergencia de la misma en caso necesario. Respecto al objetivo de mantenimiento, RiskMan analiza el impacto de las operaciones de mantenimiento de la planta • antes de su realización. Este análisis es necesario ya que la combinación de diversas operaciones de manteniento sobre la planta de manera simultánea en la misma área geográfica puede ser peligrosa. La tecnología que da soporte a la arquitectura de RiskMan es ICa. PIKMAK. El acrónimo significa Información del Proceso y Modelado del Conocimiento para Control Avanzado (de Process Information and Knowledge Modeling for Advanced Control). Los objetivos de PIKMAK se centran en el control de la calidad, cantidad y mantenimiento del proceso de producción de clinker. El objetivo de cantidad se atacó mediante la construcción del PPSI (Production Performance Synthetic Indicator) o Indicador Sintético del Rendimiento de la Producción). Respecto al objetivo de calidad el problema principal es el control de algunos parámetros que afectan al proceso de producción. Para su control se desarrolló el QDED (Quality Deviation Early Detector) o Detector Temprano de la Desviación de Calidad. El objetivo de mantenimiento consiste en la asistencia a los operadores durante los turnos nocturnos de fin se semana en los que controla la planta un solo operador. Para este propósito se desarrollo el AMOA (Alarm Manager Operator Assistant) o Gestor de Alarmas Asistente del Operador. AMOA asiste al operador en la toma de decisiones sobre si un fallo en la planta debe ser o no reparado inmediatamente. Aunque esta decisión puede parecer sencilla, en realidad existen numerosos factores a tomar en cuenta ya que el punto óptimo de operación de la planta depende del perfil de coste de la energía eléctrica para las horas y días siguientes a la detección del problema, del stock existente de clinker, del plan de producción, de la disponibilidad de repuestos para las reparaciones, etc. El octavo capítulo presenta las conclusiones y trabajos futuros a realizar en el entorno de ICa y, en particular, de ICa ORB. Como se explica, los objetivos fundamentales de la Tesis han sido completamente satisfechos. El primer objetivo consistía en la demostración de una tecnología e infraestructura que permitiese acometer con facilidad el desarrollo de sistemas de control distribuido en una variedad de dominios y que garantizase las cualidades finales del sistema final. Para ello, se ha desarrollado y utilizado el broker ICa de tiempo real tanto en grandes sistemas industriales como para sistemas empotrados distribuidos con diferentes requisitos en cuanto al comportamiento de tiempo real. El segundo objetivo fundamental consistía en el desarrollo de una metodología que permita la aplicación de ICa para la construcción de Sistemas de Control CORBA de una manera sistemática (o más bien metódica). La metodología ICm adopta el punto de vista de la Ingeniería de Dominio lo que introduce la posibilidad de o bien la construcción de sistemas específicos de control sobre ICa mediante la utilización de Ingeniería de Aplicación, o bien la construcción de entornos específicos (también sobre ICa) para el desarrollo de múltiples sistemas (de familias) de control CORBA en un dominio específico de control. La elección de una u otra alternativa depende de los objetivos del equipo de desarrollo. Entre las conclusiones de la Tesis, existen algunas que son el resultado de la evolución de los trabajos a lo largo de estos últimos años y que no hemos llamado fundamentales ya que son un producto de estas últimas. Las ventajas que el uso de la tecnología CORBA tiene para el desarrollo de sistemas de control no serían aprovechables por otros si no se realiza una labor de divulgación de las posibilidades de la tecnología. Durante los últimos años esta labor de divulgación ha sido una constante. El principal resultado de este esfuerzo ha sido la creación dentro del subgrupo de Sistemas de Tiempo Real, Empotrados y Especializados del Object Management Group (OMG) de un grupo para la especificación de interfases para Sistemas de Control CORBA. Este grupo de denomina Control Systems Working Group. La creación de especificaciones dentro de este grupo garantiza el consenso en su creación, de manera que se defiende el uso de estándares para la construcción de estos sistemas, otro de los objetivos de esta Tesis. Otro de los resultados de los trabajos realizados durante los últimos años ha sido el desarrollo de la infraestructura necesaria para la construcción de Sistemas de Automatización de Subestaciones eléctricas mediante la aplicación de la metodología ICm de esta Tesis. La mencionada infraestructura corresponde a la ingeniería de dominio contenida en el estándar IEC-61850 y al conocimiento del mismo de compañías del sector. Esta infraestructura es la primera implementación a nivel mundial del estándar sobre tecnología CORBA (ICa). También existen terceros que ya se han beneficiado del conocimiento sobre CORBA obtenido durante de los trabajos realizados en esta Tesis. En este sentido, he recibido peticiones de diversas empresas para proporcionar asesoría como experto en tecnología CORBA. De esta manera, he podido asesorar a compañías de diferentes ámbitos tales como el entorno militar para algunos sistemas de la O.T.A.N., el entorno de la gestión del tráfico aéreo para el desarrollo e interoperabilidad de los futuros sistemas del Cielo Único Europeo o el entorno del control de líneas de producción mediante tecnología CORBA. El interés de estas compañías por la utilización de esta tecnología para los propósitos de control no es más que una corroboración de los resultados aquí expuestos. En este sentido, la Tesis representa una contribución de valor añadido a las políticas de la Unión Europea respecto a la estandarización, interoperabilidad de sistemas y mejora de las condiciones de vida de los europeos mediante la mejora del desarrollo de los sistemas de control distribuido. Por supuesto, el trabajo realizado abre una serie de posibilidades que es necesario explorar como parte de los trabajos futuros a desarrollar sobre los resultados de la Tesis. El primero de estos trabajos está relacionado con el desarrollo de servicios para Sistemas de Control CORBA. La disponibilidad de servicios predefinidos permitiría el uso de ICa de manera que se pueda seleccionar funcionalidad preconstruida para sistemas de control distribuido. El resultado de esta aproximación es una manera más predecible de construir sistemas de control así como la estandarización de su desarrollo. Los servicios pueden ser de muchos tipos y, en muchos casos, útiles para muchos dominios de control: servidores de datos, gestores de alarmas y eventos, servicios de históricos de datos, servicios de diagnóstico, servicios de análisis y generación de informes, servicios de integración de dispositivos, servicios para interfases de usuario, etc. El segundo de los trabajos futuros a realizar sería la construcción de infraestructuras sobre ICa para el desarrollo de sistemas distribuidos de control en dominios específicos. La idea es usar ICa ORB e ICm para construir infraestructuras de la misma manera que se ha hecho para los SAS. Mientras que los servicios del párrafo anterior tienen validez para muchos dominios de control diferentes, las infraestructuras sirven para desarrollar sistemas de control en un solo dominio. El tercer trabajo futuro a realizar sobre ICa sería la construcción de un Entorno de Control Integrado (ICe) para ICa. Actualmente, la mayoría de las herramientas de ICa están desarrolladas para su utilización desde la línea de comandos. La inclusión de entornos gráficos para el desarrollo de sistemas de control con ICa y el uso de técnicas de desarrollo rápido de aplicaciones (RAD) facilitarían mucho las tareas relacionadas con el trabajo y uso diario de ICa. Esta alternativa se está explorando por el grupo de investigación dentro del marco del entorno Eclipse y el modelo de desarrollo MDD (Model Driven Development). Finalmente, como conclusión a los trabajos futuros, se encuentran las tareas fundamentales de mantenimiento y evolución de ICa ORB. Se debe explorar la inclusión en ICa ORB de funcionalidad para el soporte de planificación dinámica de tareas. La idoneidad (en la práctica) de funcionalidad de este tipo está por determinar. También es necesario desarrollar diferentes plug-ins para soportar los protocolos de comunicación más utilizados en la industria. Esta puede ser una labor específica de cada dominio en muchos casos por lo que es posible que esos plug-ins sean parte del desarrollo de la infraestructura para un dominio. Por último, otra labor rutinaria pero de obligada realización es la migración de ICa ORB a cuantas plataformas y sistemas operativos sea posible. Para ser realmente interoperable hay que ser independiente del hardware y del sistema operativo. ICa ORB está perfectamente preparado para ser migrado a cualquier plataforma gracias a la capa de portabilidad existente en su arquitectura. This page has been intentionally left blank This page has been intentionally left blank Abstract Many control systems of today are heterogeneous Distributed Control Systems (DCS). They are also complex systems as a great number of software and hardware elements are composed together to perform global functions at the plant level. The number of reasons for system distribution is wide: the control computation must be close to the process as communications delays can not be afforded, the plant is in itself distributed in a geographical area, there is a need of reliability and fault tolerance for the control system and many others. These reasons, that become requirements, make distributed control systems difficult to build and the software developed for them keeps increasing its costs while, at the same time, their hardware costs are decreasing. This Thesis presents a research effort to address these problems of construction of distributed control systems. The Thesis advocates the use of a common technology (CORBA) specialized for distributed control systems in a wide span of distributed control domains. Thus, CORBA is the base upon which heterogeneity, complexity and lack of system integration can be tackled in the development of DCSs. For the research, a real-time CORBA Object Request Broker, ICa ORB, has been completely rebuilt from a previous research work in the Integrated Control Architecture (ICa) which can be denominated a long-term research effort. There is not a single line of code from the first version of ICa in its new release, so it is a fully new ORB and not just a revamp of the previous one. ICa ORB has been built according to the CORBA, minimum CORBA and real-time CORBA specifications as it is considered in this Thesis that open specifications and standards are needed in order to provide full integration among disparate systems. ICa has been devised as the common technology that is suitable for the development of either large control systems or small ones like embedded distributed systems with hard real-time requirements. In order to demonstrate the feasibility of the approach, to use a single technology for the development of DCSs in many different control domains, a wide range of control systems has been developed in the past years. Substation Automation Systems (SAS), strategic control of petrochemical plants, strategic control of cement production plants or robot distributed control systems are some of them. Along with the ICa ORB development effort, a methodology for its use in the construction of DCSs is also presented in this Thesis. The ICm methodology uses a domain engineering approach to develop control systems as product lines from the core software assets of a product family. Domain engineering is used to infer and develop those core software assets for distributed control. The assets constitute a software framework that allows developing multiple control systems for a certain control domain (e.g. substation automation systems or automotive applications) or just one-of-a-kind systems for a specific control purpose. ICa ORB and ICm are used to develop distributed control systems in a process that makes development problems of DCSs disappear or at least be greatly alleviated. When a distributed control system is built in this way it is said in this Thesis to be a CORBA Control System, which is the rationale behind its title. This page has been intentionally left blank This page has been intentionally left blank Table Of Contents Chapter 1 Introduction ..........................................................................................1 1.1 Introduction ....................................................................................................3 1.2 Software Architecture.....................................................................................5 1.3 Object Technology .........................................................................................6 1.4 Distributed Object Frameworks......................................................................9 1.5 Distributed Control Systems.........................................................................11 1.5.1 Execution Environment.........................................................................12 1.5.2 Complexity ............................................................................................14 1.5.3 Real-Time Features ..............................................................................15 1.6 CORBA Control Systems.............................................................................15 1.7 Need of Integration.......................................................................................17 1.8 The Importance of Standards........................................................................18 1.9 Thesis Objectives..........................................................................................18 1.10 Thesis Structure ............................................................................................19 Chapter 2 Industrial Control Systems and Protocols........................................21 2.1 Distributed Process Control..........................................................................23 2.1.1 Types of Distributed Systems to Control ..............................................24 2.1.2 Layers of Control..................................................................................25 2.1.3 Process Control Elements ....................................................................27 2.1.4 Sensors, Controllers, Actuators............................................................28 2.1.5 Industrial Communications Protocols..................................................31 2.1.6 Trends of Control Networks .................................................................34 2.2 Commercial State Of The Art DCSs ............................................................35 2.2.1 Honeywell Experion PKS .....................................................................35 2.2.2 Airbus 380 Cabin Pressure Control System .........................................37 2.2.3 AVE Power Distributed Control System...............................................38 2.3 Critique Statement ........................................................................................41 Chapter 3 Distributed Component Middleware ................................................43 3.1 Definition of Middleware .............................................................................45 3.2 Taxonomy of Middleware ............................................................................46 3.2.1 Transaction-Processing Monitor (TPM) Middleware..........................46 3.2.2 Remote Procedure Call (RPC) .............................................................48 i 3.2.3 Message Oriented Middleware (MOM)................................................50 3.2.4 Object-Oriented and Component Middleware .....................................51 3.3 Components vs. Distributed Objects ............................................................54 3.4 Open Group’s Distributed Computing Environment....................................55 3.5 OMG’s Distributed CORBA Architecture ...................................................57 3.6 Microsoft’s Distributed Component Architecture ........................................59 3.6.1 .NET Framework ..................................................................................61 3.6.2 OLE for Process Control ......................................................................62 3.6.3 Microsoft’s View of DCOM and CORBA .............................................63 3.7 Sun’s Java Distributed Component Architecture .........................................64 3.8 Comparing Apples to Apples .......................................................................67 Chapter 4 CORBA Systems .................................................................................69 4.1 CORBA Technology ....................................................................................71 4.2 The OMG......................................................................................................71 4.3 Brokering Architectures ...............................................................................72 4.4 Object Management Architecture.................................................................73 4.5 Services and Facilities ..................................................................................75 4.6 Real-Time CORBA ......................................................................................78 4.6.1 Fixed-priority Real-Time CORBA ........................................................79 4.6.2 RT-ORB ................................................................................................80 4.6.3 Real-time POA......................................................................................80 4.6.4 CORBA Priorities .................................................................................81 4.6.5 Real-time Current Interface..................................................................82 4.6.6 Real-time CORBA Priority Models and Transforms ............................82 4.6.7 Synchronization ....................................................................................84 4.6.8 Handling Concurrency .........................................................................84 4.6.9 Handling of Connections ......................................................................86 4.6.10 Invocation Timeout ...............................................................................88 4.6.11 Protocol Configuration.........................................................................89 4.6.12 Real-Time Scheduling Service ..............................................................90 4.7 Dynamic-Scheduling Real-Time CORBA ...................................................90 4.7.1 Distributable Thread ............................................................................91 4.7.2 Real-time CORBA Scheduler................................................................93 4.7.3 Advice about Real-Time CORBA..........................................................94 4.8 Minimum CORBA .......................................................................................94 4.9 Fault-Tolerant CORBA ................................................................................95 4.9.1 Grouping Abstractions .........................................................................95 4.9.2 Architectural Overview.........................................................................97 4.10 Data Distribution ..........................................................................................98 4.11 CORBA Component Model .........................................................................99 4.11.1 What is a CORBA Component?..........................................................100 ii 4.11.2 Limitations of the CCM for Distributed Control ................................101 4.11.3 The COMPonent Approach for Real-time and Embedded (COMPARE) Project 102 Chapter 5 Using CORBA in Control Systems..................................................105 5.1 CORBA Weaknesses..................................................................................107 5.2 CORBA Control Systems Domains ...........................................................110 5.3 The Integrated Control Architecture (ICa) ORB ........................................110 5.4 Levels of Requirements ..............................................................................112 5.5 Sources of Non - Determinism ...................................................................114 5.6 General ICa ORB Performance Improvements ..........................................116 5.6.1 Client Stubs and Server Skeletons ......................................................117 5.6.2 Marshalling ........................................................................................117 5.6.3 Memory Management .........................................................................118 5.6.4 Transport Buffering and Delays .........................................................118 5.6.5 Thread Dispatching ............................................................................119 5.6.6 Request Dispatching ...........................................................................119 5.7 ICa ORB Specific Features for Hard Real-Time Systems..........................122 5.8 CORBA Communication Model ................................................................122 5.9 CORBA Control Loop................................................................................123 5.9.1 CORBA Controllers............................................................................124 5.9.2 Timing Constraints .............................................................................126 5.9.3 Loop Timing........................................................................................127 5.10 Event Triggered vs. Time Triggered Systems ............................................128 5.11 Distributed Synchronization .......................................................................129 5.11.1 Clock Driven / Time Triggered Systems .............................................130 5.11.2 RT Ethernet / Event Driven Systems...................................................130 5.12 The Notion of Time ....................................................................................130 5.12.1 Driving the System from the Communications Layer .........................131 5.12.2 Interface Extensions to the Extensible Transport Framework ...........131 5.12.3 Where to Ask for the Time in the Pluggable Framework? .................132 5.12.4 Particularization for the ETF Submission..........................................134 5.12.5 Asking the Time from an ICa Application ..........................................137 5.12.6 Life as a RTObject ..............................................................................138 5.13 Periodic, Aperiodic and Sporadic Activities ..............................................139 5.14 Timestamping .............................................................................................141 5.15 HRTC Protocol Statistics............................................................................142 5.16 C++ Source Code Example ........................................................................143 5.17 The TTP Transport Plug-in.........................................................................145 5.17.1 Main Constraint for a HRTC TTP Transport Plug-in ........................145 5.17.2 GIOP Messages Written into CNI ......................................................145 5.17.3 Pluggable Data Interface ...................................................................147 iii 5.17.4 Active Server.......................................................................................148 5.17.5 Standard CORBA + TTP/C Application.............................................149 5.17.6 Implemented Option for TTP/C ..........................................................150 5.18 The RT-Ethernet Transport Plug-in............................................................151 5.18.1 Switched Scheduled Ethernet..............................................................152 5.18.2 Real-Time Channels ...........................................................................153 5.18.3 RT Ethernet Pluggable Transport ......................................................153 5.19 Platforms for CORBA Control Systems.....................................................155 5.19.1 Embedded industrial PC Boards ........................................................155 5.19.2 Networked Device Servers..................................................................155 5.19.3 Control Units ......................................................................................156 5.19.4 Time-Triggered Hardware .................................................................156 5.20 The ICa CORBA IED.................................................................................157 Chapter 6 A Methodology for CORBA Control Systems ...............................161 6.1 Need of a Methodology ..............................................................................163 6.2 Domain Engineering for CORBA Control Systems ...................................164 6.3 Methodology Overview..............................................................................167 6.4 CORBA Control Systems Domain Analysis ..............................................169 6.4.1 CORBA Control Systems Scope..........................................................169 6.4.2 ICa Domain Analysis..........................................................................170 6.4.3 Steps of CORBA Control Systems Domain Analysis with ICa............172 6.4.4 Example of ICa Domain Analysis for SAS..........................................174 6.5 CORBA Control Systems Domain Design.................................................177 6.5.1 ICa Domain Design and Implementation ...........................................177 6.5.2 Steps of CORBA Control Systems Domain Design and Implementation with ICa 178 6.5.3 Example of ICa Domain Design for SAS............................................180 6.6 CORBA Control Systems Implementation.................................................185 6.6.1 Steps of CORBA Control Systems Implementation with ICa..............186 6.6.2 Example of ICa Control Implementation for a SAS Application ........187 6.7 Pattern and Architecture Exploitation ........................................................191 Chapter 7 Exemplar CORBA Control Systems...............................................197 7.1 Summary List of Funding for this Work ....................................................199 7.1.1 IST-2001-37652 Hard Real-Time CORBA (HRTC) ...........................199 7.1.2 IST-1999-10251 Distributed Object Telecontrol Systems and Networks (DOTS) 200 7.1.3 ESPRIT Project 22130 – Distributed Information Technology for Strategic Multiobjective Process Control (DIXIT).............................................202 7.2 Exemplar Applications of ICa ....................................................................203 7.2.1 Process Control Testbed.....................................................................204 7.2.2 Distributed Robot Control ..................................................................206 iv 7.2.3 Substation Telecontrol Application ....................................................208 7.2.4 HYDRA-Vision....................................................................................211 7.2.5 Teleoperation of a Virtual Robot........................................................215 7.2.6 RISKMAN ...........................................................................................218 7.2.7 PIKMAC .............................................................................................219 Chapter 8 Conclusions and Future Work ........................................................223 8.1 Conclusions ................................................................................................225 8.1.1 Specific Thesis Objectives ..................................................................225 8.1.2 Awareness of Standardization Bodies ................................................226 8.1.3 Domain Engineering of Substation Automation Systems ...................227 8.1.4 Results for Third Parties.....................................................................227 8.1.5 Research Support................................................................................228 8.2 Future Work................................................................................................229 8.2.1 CORBA Control System Services .......................................................229 8.2.2 Development of Domain Specific Control Frameworks .....................231 8.2.3 The ICa IDE .......................................................................................231 8.2.4 Extending and Maintaining ICa ORB.................................................232 8.2.5 ICa Pattern Language ........................................................................233 GLOSSARY ..............................................................................................................235 Bibliography..............................................................................................................265 v List Of Figures FIGURE 1: A COMPLEX PROCESS PLANT..........................................................................4 FIGURE 2: A CONTROL OBJECTS HIERARCHY IN ICA ......................................................8 FIGURE 3: CONTROL LAYERS .......................................................................................13 FIGURE 4: HYBRID ARCHITECTURE OF COMPLEX CONTROLLERS .................................26 FIGURE 5: A MULTIFUNCTION IED FROM GE ...............................................................29 FIGURE 6: A TTP DRIVE-BY-WIRE BOX FROM TTTECH................................................30 FIGURE 7: A SNOW GROOMER BASED ON A TTP NETWORK (FIGURE AUTHORING OF TTTECH) ..............................................................................................................30 FIGURE 8: GEOGRAPHICAL DISTRIBUTION OF FIELD BUS USAGE ..................................31 FIGURE 9: PROCESS PLANT INTEGRATION ....................................................................32 FIGURE 10: TDMA BUS ACCESS SCHEME IN TTP.........................................................33 FIGURE 11: TRENDS OF CONTROL SYSTEM NETWORKS ................................................35 FIGURE 12: PLATFORM ARCHITECTURE OF EXPERION PKS (FIGURE AUTHORING BELONGS TO HONEYWELL) ..................................................................................36 FIGURE 13: AIRBUS 380 CABIN PRESSURE CONTROL SYSTEM SCHEMA ........................38 FIGURE 14: AVE POWER SUBSTATIONS DISTRIBUTED CONTROL SYSTEM (FIGURE AUTHORING BELONGS TO AELEC) ......................................................................39 FIGURE 15: TYPES OF MIDDLEWARE SERVICES ............................................................46 FIGURE 16: TRANSACTION PROCESSING MONITOR ......................................................47 FIGURE 17: REMOTE PROCEDURE CALL MIDDLEWARE ...............................................49 FIGURE 18: MOM ARCHITECTURE ...............................................................................51 FIGURE 19: GENERAL ARCHITECTURE FOR OBJECT-ORIENTED MIDDLEWARE .............52 FIGURE 20: CLASSIFICATION AND TYPES OF COMPONENTS..........................................55 FIGURE 21: COM COMPONENTS RUNNING IN INTRA-HOST PROCESSES........................59 FIGURE 22: DCOM INTER-HOST ARCHITECTURE .........................................................60 FIGURE 23: THE I / O DRIVER DATA ACCESS PROBLEM ................................................62 FIGURE 24: RMI ARCHITECTURE ..................................................................................65 FIGURE 25: THE J2EE ARCHITECTURE .........................................................................66 FIGURE 26: THE CORBA OMA....................................................................................74 FIGURE 27: REAL-TIME CORBA EXTENSIONS PROVIDE STRONG CONTROL OVER APPLICATION RESOURCES ON THE CLIENT AND SERVER SIDES ............................79 FIGURE 28: CORBA PRIORITIES AND MAPPING TO OS ................................................81 vii FIGURE 29: CLIENT PROPAGATED PRIORITY MODEL ....................................................83 FIGURE 30: SERVER DECLARED PRIORITY MODEL ........................................................83 FIGURE 31: THREADPOOL MODELS, WITH AND WITHOUT LANES .................................85 FIGURE 32: ASSIGNMENT OF THREADPOOLS TO REAL-TIME POAS ..............................86 FIGURE 33: PRIORITY BANDS AND EXPLICIT BINDING ..................................................87 FIGURE 34: MULTIPLEXED AND PRIVATE CONNECTIONS IN REAL-TIME CORBA .......88 FIGURE 35: PROTOCOL CONFIGURATION IN REAL-TIME CORBA ................................89 FIGURE 36: CONTROL FLOW IN A DISTRIBUTED PROCESSING SYSTEM .........................91 FIGURE 37: DISTRIBUTABLE THREAD WITH NESTED SCHEDULING SEGMENTS .............92 FIGURE 38: FAULT TOLERANCE DOMAINS ....................................................................96 FIGURE 39: ARCHITECTURAL OVERVIEW OF A FAULT TOLERANT CORBA SYSTEM ....97 FIGURE 40: EXTENDED CORBA COMPONENT AND PORTS ........................................101 FIGURE 41: AREAS OF WORK FOR THE RT/E COMPONENT FRAMEWORK ..................103 FIGURE 42: A TYPICAL AUTOMATION INFRASTRUCTURE ...........................................107 FIGURE 43: CORBA AIR-TRAFFIC FLIGHT CONTROL SYSTEM (ATC SYSTEM STRUCTURE FROM [LIU 00]) ..............................................................................113 FIGURE 44: POINTS OF PREDICTABILITY IMPROVEMENT ............................................115 FIGURE 45: RESOURCE CONTROL FEATURES IN ICA ...................................................116 FIGURE 46: COMPARISON OF N AND LOG N VERSUS N ................................................121 FIGURE 47: DISTRIBUTED CORBA CONTROL LOOP ..................................................124 FIGURE 48: EMBEDDED CORBA CONTROL LOOP......................................................125 FIGURE 49: CONSTRAINTS OF LOOP TIMING ...............................................................127 FIGURE 50: THE ICA OCI TRANSPORT PLUG-IN ARCHITECTURE ................................133 FIGURE 51: SOURCE OF TIME INFORMATION IN ICA ...................................................134 FIGURE 52: GIOP MESSAGES COPIED DIRECTLY TO CNI MEMORY ............................146 FIGURE 53: PLUGGABLE DATA INTERFACE .................................................................147 FIGURE 54: THE RT-ETHERNET APPROACH................................................................154 FIGURE 55: A PC104 AND A HDD3.5'' FORM FACTOR INDUSTRIAL PC BOARDS (FROM LANNER ELECTRONICS) .....................................................................................155 FIGURE 56: AXIS COMMUNICATIONS NETWORK DEVICE SERVER (FRONT AND BACK SIDES) .................................................................................................................156 FIGURE 57: A CONTROL UNIT FROM ELIOP ...............................................................157 FIGURE 58: A TTP DEVELOPMENT CLUSTER (LEFT) AND A TTP X-BY-WIRE BOX (RIGHT) FROM TTTECH ......................................................................................157 FIGURE 59: UML DEPLOYMENT DIAGRAM OF THE ICA IED ......................................158 FIGURE 60: UML PACKAGE DIAGRAM OF THE ICA IED.............................................160 FIGURE 61: SYSTEM ARCHITECTURE BUSINESS CYCLE ..............................................164 FIGURE 62: INTEGRATED CONTROL METHODOLOGY PROCESS ..................................168 FIGURE 63: DOMAIN ENGINEERING AND ANALYSIS WITH ICA ...................................171 FIGURE 64: PHASES OF ICA DOMAIN ANALYSIS .........................................................174 FIGURE 65: GENERAL VIEW OF THE SAS APPLICATION DOMAIN. ..............................175 viii FIGURE 66: GENERAL STRUCTURE OF A SUBSTATION AUTOMATION SYSTEM AS SHOWN IN THE IEC 61850 STANDARD [IEC 01C]. LEVEL 0 = PROCESS, LEVEL 1 = BAY, LEVEL 2 = STATION. ...........................................................................................176 FIGURE 67: PHASES OF ICA DOMAIN DESIGN AND IMPLEMENTATION........................181 FIGURE 68: SAS LEVEL 1 SUBSYSTEM IDENTIFICATION ............................................182 FIGURE 69: THE ACSI IMPLEMENTATION FROM IEC-61850......................................184 FIGURE 70: PHASES OF CORBA CONTROL SYSTEMS IMPLEMENTATION ..................187 FIGURE 71: THE REE SUBSTATION IN VILLAVICIOSA DE ODÓN ................................189 FIGURE 72: PHYSICAL LAYOUT FOR INTERLOCKING TESTS ........................................192 FIGURE 73: INTERLOCKING OPERATION REJECTED.....................................................193 FIGURE 74: INTERLOCKING OPERATION PERMITTED ..................................................194 FIGURE 75: HARDWARE SETUP FOR THE PROCESS CONTROL TESTBED ......................204 FIGURE 76: THE PROCESS CONTROL DEMONSTRATOR................................................206 FIGURE 77: ROBOT CONTROL TESTBED SETUP ...........................................................207 FIGURE 78: THE IRB-2000 ROBOT READY TO CATCH THE THROWN BALL. NOTICE THE TWO CAMERAS ON THE WALL. ...........................................................................208 FIGURE 79: IEC-61850 STANDARD PERSPECTIVE ON FUNCTION PROVISION BASED ON LOGICAL NODES .................................................................................................209 FIGURE 80: SETUP OF THE DOTS PILOT SYSTEM........................................................211 FIGURE 81: HYDRA-VISION GRAPHICAL USER INTERFACE .........................................212 FIGURE 82: CONTROLLED EQUIPMENT AND HYDRA-VISION NETWORK ....................214 FIGURE 83: THE SIMULATED ANTHROPOMORPHIC ROBOT ARM .................................216 FIGURE 84: MULTIPLE SYSTEMS CAN BE TELEOPERATED WITH A SINGLE IDL INTERFACE .........................................................................................................217 FIGURE 85: RISKMAN USER INTERFACE...................................................................219 FIGURE 86: PIKMAC USER INTERFACE ......................................................................220 FIGURE 87: CONTROL SYSTEMS WG WEB PAGE AT OMG'S SITE ...............................226 ix List Of Tables TABLE 1: GENERAL SYSTEM QUALITY ATTRIBUTES ACHIEVABLE BY ARCHITECTURE 14 TABLE 2: CLASSIFICATION OF MIDDLEWARE FEATURES ..............................................67 TABLE 3: REAL-TIME CORBA PRIORITY MODELS ......................................................82 xi xii Chapter 1 Introduction 1 1.1 Introduction Nowadays, most control systems are heterogeneous distributed systems. They are also complex systems as a great number of different hardware and software elements are combined together to provide global functions. The technical reason for heterogeneity is that specialized equipment performs better some functions of a system; sensorreading and actuator control, process management and enterprise management are placed at different levels of the distributed control system and perform different tasks. In the case of control systems, the main reason for distribution is that computing must be close to the process in order to ensure prompt reaction to process changes (e.g. lower-level control loops). Figure 1 shows this situation in a complex process plant. However, there are many other reasons for doing this: • availability of suitable embeddable processors, • timing requirements that forbid communication due to latencies, • need of increased levels of performance that is achieved through parallelism, • simplification of construction and maintenance tasks through modularity, • reduction of cost and time-to-market by means of component-based reuse, • integration of legacy systems • availability of task specific software platforms • Reliability and fault tolerance. The example of the process plant can be easily translated to other domains where distribution and heterogeneity are key factors. The reader may think of modern warfare systems where information flows from many different sources (radar, satellite, targetting, engine control data, etc.), and is drained by a similar amount of sinks (troops, avionics, combat systems, positioning systems, etc). Another example is in the automotive industry where modern ABS systems have a dedicated computer in each wheel exchange braking information with the rest. Additionally the suspension, power management, and engine control systems work together with the ABS to keep the automobile under control. While these systems are common these days, what are the common aspects among all of them? They are difficult to build. While hardware costs keep decreasing over the years, the software that runs on it becomes more and more complex and the effort and capital investment to build it keeps rising. The problem has several facets two of which, 3 distribution and heterogeneity, are of surmounting importance in order to properly build the system. Safety MIS Enterprise Network Business Management Data Storage Process Control Process Operation Control Network Process Management Field Configuration Fieldbus Sensing and Acting Field Management Continuous Process Plant Figure 1: A complex process plant Embodied in this document, a research on a common technology (CORBA) to address these problems in the field of distributed control systems is carried out. Control systems greatly vary from the very small to the very large and their requirements also vary greatly from the very time-tight systems to others where time is less critical like those related to the Management Information System of the plant. Generally speaking, the best way to get rid of those problems is to make them disappear at the system development level. A common abstraction for a wide variety of systems is needed. Making the abstraction common means that it is platform-independent, so perceived heterogeneity can be actually removed. To avoid the complication emerging from distribution we should develop the system as if it were a monolithic system. These are precisely the ideas behind the Object Management Architecture (OMA): to provide abstraction above low-level detailed platform-problems and to provide distribution and heterogeneity transparency to systems in an object-oriented way. This is achieved by providing concrete object models based on the OMA; namely CORBA. Concrete implementations of the CORBA specification hand to system builders a homogeneous platform-independent infrastructure for system development. 4 But whereas the homogeneous CORBA abstraction has proven useful regarding some domains, the concrete object model of the CORBA architecture is not good enough in some situations. CORBA was designed with the objectives of flexibility, interoperability and reduction of complexity without incurring in a significant loss of performance. Unfortunately, there are system demands, like those of control systems, and requirements, as those related to real-time, embedded, and fault tolerant systems, that cannot be overlooked to build a successfully working system. A careful analysis and implementation of the supporting framework technology for these systems is then needed as it must bear the basic properties the final system must exhibit. To meet all these requirements, a disciplined approach is needed and this Thesis relies in three areas of computing science to achieve them. • • • Software Architecture: As the guiding discipline to achieve system properties that are architectural in nature. Object Technology: As the technical support for building abstract representations of the world. Distributed Object Technology: To provide the basic infrastructure upon which control architectures are implemented. 1.2 Software Architecture As distributed control systems are software intensive artifacts, a way to specify their architecture in the same way other engineering disciplines do is needed. By means of Software Architecture, critical properties of the final system are defined, which will be almost impossible to modify in later stages of development. If control systems were small or if their properties as software system were easy to achieve we would not need to worry about Software Architecture. Coding algorithms is probably all we would need to do. Although this might seem a “light-headed” statement, it is not. All of us know examples of this type of development. Unfortunately, distributed control systems are neither small nor their requirements are easy to achieve. When systems begin to grow, there is no other way to success than to divide the functionality of the system. In other words, we need to define modules that form part of the system. Defining modules and the relationships between them will also provide us with the ability of modifying them independently from the rest. When the problem is that the features are hard to achieve, we also need architecture to be able to focus in the design of specific parts of the system. For instance, how to design a module with a fast cyclic digital input scan. 5 Software Architecture is important because it appears early in the life cycle of the system and helps to make decisions about its composition, structure and the relationships between all the elements that compose the system. These decisions do not only affect the properties of the system but, also affect the capability of the developing organization in its market niche. Software Architecture influences how a company can compete in the market or what the time-to-market for a family of products will be. This means that the business organization must not only focus on the technical properties of a system’s Software Architecture but also in others that affect how business in done. The software engineering team must then create a balance or define a trade-off between the qualities of the system from both points of view, technical and those related to business. In fact, and only from the technical point of view, defining a Software Architecture will help to make the trade-off about the technical capabilities of the system; availability, dependability, performance, etc. All these are qualities of the system that can only be specified by architecture. If an architecture fails to meet any of the architecture-related attributes of the system, then a successful system cannot be built. Hence, the importance of enforcing the use of sound architectures for Complex Control Systems. 1.3 Object Technology The control engineering community is still reluctant to a change in the core technology that drives their systems. Understandably, they build and operate control systems of great complexity, whose requirements are difficult to meet, and in consequence they stick to technologies that have proven to solve most of their problems over decades now. Nevertheless, everybody perceives the value and the advantages that Object Technology (OT) has to offer, precisely in an area whose properties of complexity and heterogeneity can be suitably addressed by it. Objects can be thought of as abstract representations of real-world or other entities that are independent and able to manage by themselves. Functionality is achieved by the services the objects provide and information is exchanged by message passing. A first key-point in the object concept is that they encapsulate and hide their state which can only be accessed or modified by the services provided by the object. Thus, internally modifying an object does not affect other objects in the system which is the cornerstone for isolating and managing change in the system. Other objects can request the services of any other object in order to make the system progress towards its objective. 6 The second key-point for objects is the concept of abstraction. Through objects we can capture the essential properties of any system entities, in an attempt of making things plainly understandable. Deciding which entities are represented by objects and which are not often becomes a tough decision which will affect the overall system properties. Further, some objects might just be data-structures while others may provide computation services only, as to provide only behavior to the system. At the end, the level of abstraction depends on the type of applications being developed. The real benefit is that we have representations of notions our minds can easily manage. These two key-points fit well into the Software Architecture concept of division of functionality and lower the concepts closer to the implementation of the system. This is why Object Technology is so interesting in this research. It allows us to translate Software Architecture concepts to the implementation field. This apparently simple concept of objects, gives system developers a huge advantage over the way systems can be constructed. As mentioned before, objects are independent entities and as such can be understood as standalone parts of a system. This provides a way to handle complexity in a divide and conquer style or in an approach to system decomposition. It also provides ground for reusability, allowing developers to build code with a long life expectancy. Reusability is the best known way to cut down development costs and to shorten the time needed to build production systems. For automation industries, the possibility of reusing objects as part of their systems redirects the organization of their development core processes to develop software in product lines. This way of building systems allows the developers to share a common architecture for a range of similar systems which bear a set of common features and are built from a collection of shared software assets. The companies that evolve software product lines have largely optimized the way their software control systems are constructed. The costly investment made in system architecture development pays off as the same architecture can be reused multiple times. They have a library of software assets that can be employed many times in a set of products that share common characteristics. This is important as common control systems are expected to lend their service over decades, not to mention the effort and capital needed to build them. Objects can be mapped to real-world entities that make full sense to human beings. Further, inheritance can be employed to build hierarchies of objects of increasing complexity in a step-by-step approach (not iterative). This is just another way to deal with complexity and heterogeneity. Child objects provide slightly specialized behavior so the increase of complexity for each generation in the hierarchy is not too large to be understood. Complex behavior can be obtained from ascendants by creating descendant objects with different properties, giving birth to hierarchies of more refined or specialized objects. Figure 2 shows a hierarchy of objects to provide 7 complex control behavior built from simpler objects. Notice than in the figure, there is an orange shadow on the outer rim of each circle that represents a control object. This shadowed area is the object’s interface and it is the only way the services of the object can be requested from other objects. As will be explained later, objects can expose different types of interfaces that can be classified according to the type of service they provide. One of the activities the control systems industry is facing is the shift to object technology. They have always claimed not to use object technology for a matter of efficiency. Although this reason may still hold for some systems which must be developed in machine code, its number is greatly inferior to the remainder of systems that can benefit from object technology. One of the reasons of the availability of this change is the evolution of hardware performance. The market for hardware devices offers ever increasing performance and features for devices are always decreasing in cost. Education is another factor that is obliging manufacturers to make the transition to object technology. It is becoming harder to find qualified personnel that knows anything else but object technology as schools and Universities worldwide mostly provide teaching in this area. ICa Core Agent ICa MT Agent ICa KB Agent ICa RT Agent ICa FT Agent ICa FL Agent ICa PLC Agent But objects alone cannot provide the kind of ICa Fuzzy ICa RiskMan service control systems need. Objects need to Control Agent Agent be combined to provide the service at the global level in a distributed way and this must be done according to the set of requirements control systems need. Composing distributed objects ad-hoc cannot be considered a sound practice as the middleware code will be deeply hooked to the application. For the next system to develop, new code will need to be written to achieve the same functions the previous middleware did. This practice poses all kinds Figure 2: A control objects of problems to developers as they always hierarchy in ICa perform the same tasks but in a way particular to every system implementation. They are not 8 making good use of the gained knowledge or at least they are not optimizing it. Instead of this approach, a middleware infrastructure based on standards can be employed to achieve the distribution of objects. 1.4 Distributed Object Frameworks According to Enslow [Ens 78], distributed systems are characterized by the following properties, • • • • • They are composed of a collection of computational resources (hardware and software), which can be dynamically assigned to specific tasks. The resources are physically distributed, and collaborate by means of a communication network. An operating system exists and is used to unify and integrate the control of the components. The distribution of the computational resources is transparent, and services may be requested specifying only their name. It is not necessary to know the location of the resource, just how it is named. The operation of the hardware and software resources is characterized by a coordinated autonomy. This definition by Enslow in 1978 was one of the first ever done for distributed systems. There are other definitions which have been formulated later but the content of the definition is basically the same. For Tanenbaum [Tanenbaum 95], a distributed system is a collection of independent computers that appear to the users of the system as a single computer. Coulouris [CDK 94] gives a more detailed definition of what a distributed system is. “A distributed system consists of a collection of autonomous computers linked by a computer network and equipped with distributed system software. Distributed system software enables computers to coordinate their activities and to share the resources of the system -- hardware, software, and data. Users of a well-designed distributed system should perceive a single, integrated computing facility even though it may be implemented by many computers in different locations”. In a broad sense, an intuitive yet powerful definition is that of Burns [Burns 97], 9 “A distributed computer system is defined to be a system of multiple autonomous processing elements, cooperating in a common purpose to achieve a common goal”. For our work and in the CORBA Control Systems point of view, this notion of distributed system provides a clear common understanding of what a distributed control system is, a collection of elements collaborating to run a plant, with certain real-time properties, efficiently. As to the requirements demanded from distributed systems they are much the same of any other system. We would like our system to be efficient, flexible, robust, scalable, dependable, etc. While these requirements are common requirements for most systems, in the case of distributed systems they are harder to achieve. The reason for this is that distribution raises the complexity of the system. The distributed system architecture has to account for the communications system to be used that will affect all the properties of the system. Think for example in the real-time properties that must be met and in the latencies that a certain communication mechanism will impose by its nature onto the system. This is a problem that would not need to be treated if the system were a monolithic one. To ease the pain of developing the underlying architecture’s support for distribution, distributed object frameworks are relied on to implement this task. One of the primary objectives of this Thesis is to effectively demonstrate that there is a common technology that can be employed to develop distributed control systems at all levels; from the embedded system level where decisions are made in milliseconds to the strategic level where production planning is scheduled at a much larger time granularity (weeks, months). A distributed object framework is valuable because it is a common abstraction for functionality needed in all types of distributed systems. This means we can reuse a specific implementation of that abstraction in multiple systems, providing leverage to the work of constructing the infrastructure needed for distribution. The question is whether there is or can be constructed such a framework that enables us to implement all layers of a complex control distributed system. The framework used for the development of CORBA Control Systems should be object-oriented. There are other types of frameworks for distributed systems such as the message passing or transaction processing frameworks which are not well fit for the control purpose. The fundamental reason for which an object approach is needed is software complexity. Complexity is the effect of human being limitations. A distributed object-oriented framework helps us solve the problem of complexity in three ways. 10 • • • Object-oriented frameworks support the principle of abstraction. Object oriented frameworks support the principle of hierarchical development. Object oriented frameworks support the principle of modularity or decomposition. An object oriented framework supports abstraction by decoupling the interface of the object from the implementation of the object. This concept must not be compared with that of abstract types in the C++ language (similar but no the same) not the abstract keyword in OMG’s IDL. Through hierarchical development we can distinguish between the general properties of things and the properties of a specific kind of things. For instance, all controllers have the general properties of reading inputs and calculating outputs but only Fuzzy controllers implement fuzzy control algorithms. By inheritance we can make commonality explicit and increase system functionality in a simple manner. Decomposition and modularity are inherently supported by Distributed Object Frameworks as systems specifications are built from a composition of objects. 1.5 Distributed Control Systems Distributed Control Systems are a special case of distributed systems by its own nature. They have very special properties, take heterogeneity for example. Although heterogeneity can be controlled in some distributed systems (e.g. a banking system), in the case of distributed control systems heterogeneity is almost impossible to limit. A large distributed control system implements thousands of functions and is built around equipment from multiple vendors. Further, this type of systems is designed to service the plant for long lifetimes (15 years or more) and obviously technology changes over time. This results in the introduction of new heterogeneity as in the lifetime of the plant some systems will be replaced by newer ones while others will simply be maintained. If the complexity of building a heterogeneous distributed system is considered along with the complexity of building a large control system, a development problem of overwhelming complexity is posed to system builders. Distributed Object Frameworks such as CORBA are very effective in dealing with the problem of heterogeneity, making transparent for developers the hardware or software nature of the components of a distributed system. However, there are other types of requirements that are dealt with in an ad-hoc way at the time being. These requirements are those associated to the real-time properties of the control system. If 11 an object framework is used for the construction of distributed control systems then the framework must provide the necessary tools to control its real-time behavior so the whole application can be deployed coherently. In this Thesis, some adaptations of the current real-time CORBA specification have been used to take control over sources of non-determinism in the CORBA object framework. To fixate the scope of CORBA control systems, the execution environment and the real-time and complexity characteristics of a controlled plant are presented below. 1.5.1 Execution Environment Distributed Control Systems have a very specific execution environment. The objective of the DCS is the efficient exploitation of the interoperation capabilities of devices and systems in the distributed context of the plant. In this distributed scenario, there is a variety of systems than can be classified by diverse criteria. From a control point of view, a valid classification can be based on the following items. • • • functionality location performance For instance and returning to the example of the control layers of a process plant (see Figure 3), at the lower control level of the plant some Remote Terminal Units (RTU) or Programmable Logical Controllers (PLC) are devoted to real-time signal acquisition from the Intelligent Electronic Devices (IED) connected to them. A second level of RTUs acts as a communications front-end in a multi-layer networked structure. Both types of remote terminal units can be geographically scattered across different unattended locations. SCADA systems are located in control centres, far away from the origin of process signals and do not require the hard real-time capabilities of lower control layers. There are other tools such as configuration applications that can be run off-line. Above these layers, the Enterprise Management System can schedule production for the next days, weeks or months. All the layers are connected and information flows upward, downward and horizontally through systems of different vendors that use different technology. Horizontal and vertical integration is attained at a high cost. As already mentioned, a common characteristic in most industrial systems is the heterogeneity of hardware and software in the system. From the point of view of hardware, proprietary buses, such as VME or Compact PCI systems, are present in the 12 different devices of the plant. Software also presents a high degree of heterogeneity. Different communication protocols, proprietary monitors for embedded systems and software from different suppliers (i.e. SCADA software) are easily found in this type of systems. Almost always, there is also the need to integrate new software with legacy systems. User Interface MIS Strategic Control Optimization Tactical Control Plan execution Operational Control Reactivity Advanced Control Complex Loops Simple Loops Sensors & Actuators Conventional Process Control Continuous Process Plant Figure 3: Control Layers Thus, the main characteristic of the execution environment is heterogeneity and complexity. Concerning software there is heterogeneity and complexity in software architecture. This is because there is not such a thing as a software architecture for the whole system. These conditions lead to complex systems. Distributed control systems are complex not only because of their size; they are also complex because of their nature. The domain of the problem must be considered as well as its size. The first solution to complexity is decomposition. Split the problem into pieces until we can understand the complexity in our own minds. Unfortunately, this leads to heterogeneous systems if not handled with care. Other constraints such as the lack of standards adoption (proprietary solutions are a business barrier for competitors) make it impossible to plan a homogeneous system. Decomposition increases another form of complexity in the system. The isolated modules must be interconnected and information must be transmitted by new channels of communication. Communication protocols must be specified and checks for their 13 consistency must be established. This is where CORBA can put some leverage to the current scenario. The architectural framework of the OMG’s OMA helps reduce this type of complexity by establishing detailed interface specifications and a common understanding of what interoperability means. Additionally, in the case of industrial systems appears the need to control the timeliness of the system and to embed the software in limited-resource systems. In the case of real-time features, CORBA can also help dealing with these issues by facilitating the way in which the lower level real-time OS APIs are accessed. For embedded systems, support comes from the profiling of CORBA for smaller systems. It has been a long time since either control systems where based on “island CPUs” or were conceived as “islands of automation”. Distributed control systems must show qualities such as performance, reliability, availability, scalability, portability and maintainability among others (see Table 1). The lack of these characteristics cannot be afforded in industrial systems but cost effectiveness is also an issue. CORBA as a standard for Distributed Object Computing contributes to system development by isolating the developers from the boundaries of system modules. Quality attribute Architectural Architectural issues in nature? Discerned observing the system at runtime Yes Divide functionality to exploit parallelism Performance Yes Fault tolerance with redundant components Availability No Interaction with other quality attributes Functionality Not discerned observing the system at runtime Yes Modularized components Modifiability Yes Portability layers Portability Yes Loose coupling among components Reusability Yes Consistent component interfaces. Possibility Integrability of incremental build Yes Modularized components Testability Table 1: General system quality attributes achievable by architecture 1.5.2 Complexity Complexity is important because it affects the way we understand the devised systems. In the case of distributed control, it is of utmost importance as it affects the 14 robustness of the system. Whereas cost is important, distributed control systems can produce potentially hazardous effects on life or property if they misbehave. Then, it happens often than performance is sacrificed instead of robustness. Complexity is tackled by means of modularity, structuring the system into pieces than enable us to cope with system’s complexity and focus into the detailed analysis of independent modules. A technology that helps developers deal with complexity is needed by approaching system construction in a structured or modular way. 1.5.3 Real-Time Features Distributed control systems present conflicting requirements. This is due to the fact that on one side the system is split into modules which make the system bulky but easier to understand. On the other side, there are the real-time control requirements which demand an efficient implementation of the system (with regard to performance). Depending on the time constants of the controlled system, the degree of efficiency of the implementation may vary. In the case the control is very close to the process where resources are low, the introduction of infrastructures that allow modularity may hamper performance as the system needs to execute more code with the same processing resources. The real-time requirements pose an additional problem to distributed control systems development as tools for system infrastructure need to support real-time requirements and must be capable of being deployed over devices with little processing resources. 1.6 CORBA Control Systems Distributed objects are a useful technology for control systems construction because control systems are software systems that continuously interact with the reality (i.e. with real, physical objects) so the software paradigm that best fits this domain is the object-oriented paradigm. The CORBA community has been aware for a long time of the lack of suitability of the general CORBA specification for certain types of technical systems. They have also kept in mind the tremendous benefits that CORBA has brought to distributed system development. This change in the way systems are built has led engineers to think that the same concepts of abstraction and homogeneity could be applied to systems with more stringent operating conditions. These are basically systems that need to deal with the progression of time, dependability or scarce resources. 15 The efforts of the OMG community regarding the Distributed, Real-Time and Embedded (DRE) systems have been materialized in three different specifications which now form part of the CORBA specification suite. • • • Minimum CORBA specification. This is a profile of CORBA for lowresource systems. Basically, it removes from the CORBA specification those parts which are of little use to systems where most things are known at designtime. Real-Time CORBA specification. This specification builds on top of CORBA to provide the control of resources necessary to achieve end-to-end predictability in real-time systems. The specification reuses concepts and some parts of other specifications as the Quality of Service framework from the Messaging specification or the Enhanced View of Time specification. Fault-Tolerant CORBA specification. There are many applications that have a need for fault-tolerance. This specification deals with the CORBA infrastructure services that an application might request to achieve fault tolerance. The specification supports a range of fault tolerant strategies such as request retry, redirection to an alternative server, and passive and active replication. The increasing interest of the scientific / technical developer community in CORBA for technical systems have fostered a rising industry of ORB manufacturers for industrial applications. Most implementations are Real-Time CORBA ORBs built on top of the Minimum CORBA specification so as to take advantage of real-time features with a moderate use of resources. There are some issues that have not still been fully addressed as some requirements mix-up in critical systems. These systems have requirements regarding real-time, fault-tolerance and embedded characteristics that pose a difficult problem still unsolved unless treated ad-hoc by traditional systems engineering. The OMG specifications regarding real-time, embedded, and fault tolerant systems have been done individually and whereas most real-time ORBs provide minimum CORBA implementations they do not provide reliability characteristics regarding faulttolerance. This means that DRE applications can be fully built and deployed by the use of commercially available ORBs but the task of building the system becomes more problematic when requirements for predictability and dependability begin to be tangled together. This is also the situation in the case there is a decision to build the system in an ad-hoc way, so in benefit of CORBA the developer can always take the advantage of having a common abstraction for the development of DRE systems no matter the platform used. Other questions as the lack of awareness of the progression 16 of time in the CORBA abstraction (IDL language) are beginning to be dealt with so in a future, all types of systems with hard real-time constraints may come into operation using CORBA technology. Although, as reviewed, some problems still remain unsolved, the technology has established the quality of its value in a great extent of industrial applications ranging from onboard ship command and control or research facilities to software defined radio, tactical radio systems, telecom applications or avionics. CORBA has clear advantages regarding transparency (object location), independency (from platforms and languages), scalability, flexibility, cost, etc. when compared to traditional system development procedures that can be fully applied to systems with tightly-binding requirements. 1.7 Need of Integration When designing distributed control systems, the requirement for integration is always present. As of today, integration is a costly process that is solved by the introduction of legacy bridges among communication protocols or by gateways among standard protocols in the best of cases. Take for instance, the railway traction substation case. In a small substation, catenary overhead line protection devices may use MODBUS or the IEC60870-5-103 protocol to carry out serial communication from the protection devices to bay modules. The bay modules are attached to a ring-like redundant fiber optics network that uses a proprietary TDMA-based protocol to communicate all bay modules in the substation with an industrial PC. The PC is used as an OPC server which can be accessed from an operator workstation (an OPC client) via a TCP/IP network. Another PC linked to the TCP/IP network, acts as a gateway to connect the substation to a remote control center by means of the IEC60870-5-101 protocol. This is common case of a distributed control system. Integration is needed either horizontally or vertically. The exposed case only shows and example of what happens in the lower layers of Figure 3. Most of the protocols used in the example are employed to achieve horizontal integration but the overall system also includes production management and business operation levels so vertical integration is also needed. In this work, it is shown that CORBA is a technology that, being good in the upper enterprise management layers of the distributed control systems, can also be employed 17 at the lower control layers to integrate the diverse controllers and devices spread throughout the controlled process site. Process information can be obtained from any place in the controlled site and actions can be initiated to improve production with a single integration technology based upon the OMG specifications. 1.8 The Importance of Standards Many people do not realize that the world will not be the same without standards. Standards are one of the forces that have led the way of technology development until the present day and that have configured modern economy. In other words, standards shape our world. It all began about one hundred and fifty years ago when William Sellers gave his speech “On a uniform system of screw threads” in the lecture hall of the Franklin Institute, the professional society to which Sellers belonged. Before this, tools were custom-made for the clients. With standardization mass production became possible, ensuring at the same time that products had an acceptable quality level. Standardization has become today a political struggle where the strongest stakeholders try to impose their technologies (an example of this is the Profibus technology backed by Siemens). Many companies have realized that the best way to develop their business is to help to create standards so they actively collaborate in the standardization bodies. One of these bodies is the Object Management Group and one of OMG’s specifications is CORBA. OMG is the largest object software organization in the world with more that 500 company members. CORBA is a specification that has become an open de-facto standard which provides developers of distributed systems with a flexible middleware capable of integrating complex applications in heterogeneous environments. 1.9 Thesis Objectives With regard to the objectives of this Thesis, the target is twofold. • An investigation of the application of the CORBA technology has been carried out in order to tackle the mentioned problems of distributed control systems. This has been done in the scope of large control systems and in that of small embedded control systems. Extensions and/or modifications to existing CORBA specifications for real-time are proposed and implemented and the validity of the approach demonstrated in numerous examples of 18 • application (see Chapter 7). For this, a real-time CORBA ORB has been completely re-built. A methodology for the construction of distributed control systems based on the CORBA middleware is introduced. The purpose of the methodology is the reduction of cost and time in the construction of this type of systems. Of course, the final objective of the methodology is to ensure that the desired properties of the system will be achievable by implementation. The ICa ORB and the ICa methodology (ICm) are cornerstones of the Integrated Control Architecure (ICa) vision. 1.10 Thesis Structure The Thesis is organized around seven chapters. • • • • • • Chapter 1 is the introductory chapter. It explains, as the reader already knows, the intrinsic properties of the distributed control systems. Heterogeneity, complexity and the need of integration are among the most important factors to take into account. Chapter 2 is an overview of state-of-the-art distributed control systems that can be found in the marketplace by leading companies. The chapter also provides a review of the most widely used communication protocols in the industry. Chapter 3 presents the different types of middleware technology and the intended use of each of them. It is frequent to find combinations of middleware technologies in large systems. MOMs or transaction monitors at the business layers and other types of middleware technology at lower levels of control like OPC or CORBA. Chapter 4 introduces the OMG’s CORBA technology and the basic specifications relevant for CORBA Control Systems. It addresses issues as real-time, embedded development, fault-tolerance or specialized services with real-time requirements. Chapter 5 introduces the ICa ORB for CORBA Control Systems and the enhancements of CORBA specifications to deal with issues of real-time in distributed control systems. Chapter 6 contains the methodology for CORBA Control Systems development. The methodology makes considerations about architecture and 19 • • modeling and exposes a method to specify requirements, analyze, design, develop and deploy CORBA Control Systems. Chapter 7 exposes several examples where the CORBA Control Systems approach has been used. All are examples of complex distributed control systems that have been successfully developed and deployed. Chapter 8 summarizes the achievements of the performed work and develops some insights on the future trend of research and promotion of the technology. Finally, two annexes are appended to the document, one introducing a glossary of important terminology and another one including relevant bibliography. 20 Chapter 2 Industrial Control Systems and Protocols 21 2.1 Distributed Process Control This chapter deals with the actual state of the art of distributed industrial control systems. The focus is set on distributed control systems as putting leverage to the development of such control schemes by means of CORBA is the focus of this Thesis. The main problem of this type of control keeps being complexity [Åstrom 00]. The problems of performing process control are combined with those of developing a system which is software intensive and distributed. This work is not concerned about the complexity of the control algorithm itself but about the complexity of implementing system distribution for control purposes. The second problem in importance is that of integration [Alarcón 94]. It is common that distributed control systems are huge systems where multiple subsystems coexist. In fact, defining the boundary of what a subsystem is can be a matter of discussion. Many subsystems carry out a well-defined purpose independently and can be considered systems on its own. However, in modern industries, information flows to and fro subsystems for many different purposes, be them operational (production), tactical (advanced control) or strategic (administrative). Accordingly, all subsystems or systems must be integrated so the enterprise can be efficiently run from its helm. Traditionally, integration has been a source of painful headaches in all types of software involved businesses. Control has the additional problem that deals with real world things and therefore can either be dangerous or provoke loss. Integration should leave no doubts whether reliance can be put on the system or not. ICa solves this problem by providing a proven framework for integration. A third problem is that of flexibility [Bengtsson 04]. Large control systems are many times deployed under ad-hoc designs that impose a rigid structure to the business. This means that the design tried to solve current problems of the process control but did not envisioned the future needs of the business; and control does not exist without a business. Actually, business is under high pressure to adapt and react to changing market conditions. This implies that companies have to adapt their systems to new production environments and accept modifications as quickly as possible, usually before the competition. How the system is adapted, modified, enhanced to accommodate to market and production demands is another big issue and the root of enormous expenses. CORBA-like technologies are able to handle this type of situations. A fourth issue is standardization. Automation system and device (sensor, IED or PLC) manufacturers make large efforts in distinguishing the interfaces to their products and large stakeholders try to impose their “standard” technologies. CORBA [OMG 02] 23 provides a standard specification approach agreed by some 500 companies in the world. Standardizing interoperability protocols greatly overcomes the limitations of modifying or enhancing control systems while enormously reducing their cost. 2.1.1 Types of Distributed Systems to Control It is important to elucidate the background in which ICa has been developed. As with all tools that are thought with flexibility in mind, there is always a compromise to be made. The compromise made can be expressed in many forms; e.g. features versus memory footprint or others. When the work in ICa began, the initial focus was put in easing the development of large industrial control systems. Examples of such systems are cement plants or petrochemical refineries. In fact, some of the research projects in which our group has been involved over the years dealt with distributed control issues in these types of industry. After the initial releases of ICa, the idea grew in our minds that the possibilities of the broker could be expanded to other types of distributed systems. Primarily, embedded and real-time systems (this including hard real-time systems). There was an initial attempt to solve problems of large distributed systems where real-time features were not so important (either there were soft deadlines or the time constants of the controlled plants were large enough to be obviated). So ICa can be employed in different versions which interoperate and which are tailored for different systems. ICa can be used in full, minimum and real-time versions of the ORB. This way it is possible to use its full potential where there are no resource constraints or to build a minimal ORB optimized for embedded systems and to run the ORB at the same time over a hard real-time transport protocol. Notice that taking ICa to small devices and enabling it with real-time capabilities allowed to embed the broker in IEDs (Intelligent Electronic Devices) which were the last barrier in order to achieve full integration from a plant-wide point of view. Thus, it is possible to establish a CORBA bus that spans the whole industrial organization horizontally and vertically, from the administrative workstations to the field devices that run simple control loops closest to the process. In the following sections, an overview of modern automation systems is presented along with a variety of communication protocols that are used for industrial purposes. 24 2.1.2 Layers of Control When complex systems are designed, there is more to it than simply concentrating on the functional role the system must perform. In the case of complex control, there are multiple tasks that must be performed which can be logically classified into different levels or layers. These tasks range from the most basic control loops to the planning and scheduling of plant operations or production. Instead of building a system in which all functionalities are included in an unordered way, structure in given to the system in the form of layers [Gamma 94]. Thus, the control system is logically composed of layers that form the logical architecture of the control system. The layered architecture of the system is useful because it reduces complexity as functional parts of the system can be grouped together and developed independently. Layers minimize coupling because communication is limited to neighbor layers and at the same time the layer structure enhances reusability. Layers offer generic services that are used by upper layers so in the simplest case, each layer only needs the services of the layer immediately below it. In the case of control systems, lower layers also need to send information to the higher level layers. This information can be forwarded in the form of events which are managed by event handlers. At the implementation level this can be achieved by means of variations of the Observer pattern which enforces a publish / subscribe mechanism for the events of lower layers. Whether this mechanism is reliable or not is a matter of design and implementation. As said, layers of distributed control systems are a consequence of addressing complexity. Complexity comes also from the different types of information being handled in the system. Lower layers deal with signals closely related to process control as sensor related data and set-points whereas in the upper layers more elaborated information for production planning or emergency management is needed. This is well known in data warehousing where operational data are clearly differenced from those that are tactical or strategic. Regarding the number of layers, there is not a predefined number that fits all systems. Layers are everywhere. In the smallest distributed systems there are lots of layers if the system is carefully observed. Take for instance an operating system or a communications protocol stack where increasingly complex services are built atop simpler ones (e.g. FTP atop TCP/IP). If focus is put just on the control system itself, there is a great range in the number of layers from system to system. Small embedded systems can be run with a single layer as its purpose is to efficiently control a process probably in hard real-time (e.g. an ABS system) while in the most simple case the 25 pervasive three layer approach can be used (e.g. the planner, executive and functional layers of NASA’s robots). It is important to notice that the “universal” three layer approach may not correspond to system architecture from the control system point of view. The three layer approach may only fit the system development point of view in which a conceptual separation between data-source, control logic and presentation layer has been made. But this approach does not necessarily fit the control need of any individual industry. In fact, there are different points of view to the system architecture. There is a layered structure that belongs to the control system architecture (e.g. simple loops, complex loops, tactical control, strategic control) and another layered structure that guides how each layer is implemented. This last layered structure is where the pervasive three layer architecture approach mostly belongs. So a complex control system architecture style is heterogeneous. That means that several architectural styles can be employed in a system depending on which part of it attention is paid [Bass 98]. In the common case, control systems show a layers-oflayers architecture, some are good to handle control complexity while others are good to control development or implementation complexity. Figure 4 shows a possible hybrid architecture for a control system. Control Layers Perspective Development Layers Perspective Layers Control System Hierarchical Heterogeneity In System Architecture Figure 4: Hybrid architecture of complex controllers 26 2.1.3 Process Control Elements There is a large amount of process control elements available in the market today. The elements to be used in any singular application basically depend on the process to be controlled. This way industrial control systems are organized into areas of application. For instance, there are specific industries for chemical process control, power and utilities, automotive, semiconductor, water, military, aerospace, etc. Although the basic structure of control is largely similar in all of them, all industries have their own peculiarities that have given way to specialized providers of control systems. In the most basic configuration of distributed process control, the main elements are the sensors, the actuators, the controllers and the communications between them. It is important to notice communications as a differentiated element of process control as jitter, latency, transmission delay, queuing delay, etc. in communications pose their own challenges to process control [Liu 00]. In a real world plant, process control is somewhat more complex that the simple model of sensor-actuator-controller. This model is useful for basic loops but does not fit well in the distributed environment of a plant (e.g. a chemical plant) where different processes are interdependent and so are their process controls. In this type of environment appears an upper layer of control dedicated to Human Machine Interfaces (HMI) and Supervisory Control And Data Acquisition. The HMI mission is to allow plant operators to check process and control elements status from anywhere in the plant and to help them adjust the control settings for the process. In doing this, the HMI and SCADA isolates operators from low level details of control by means of simplified display views of the process where only relevant data is shown. But also, in a simple real-world environment, some of the process information must be shared at the business administration layers and, on the other way, information from administration must go down to the process (e.g. to adapt production to client orders). This means that the process control must also be integrated or must interface with Manufacturing Execution Systems or MES. MES are software systems that track and manage all aspects of a job in real-time while it is in process (a.k.a. execution phase). 27 While MES takes care of resource scheduling, capacity planning, process management or data collection from computerized instruments in the short term there are also control needs that must be addressed in the medium or long term. This is the mission of Enterprise Resource Planning (ERP). An ERP incorporates to the industry a higher layer of control which is specialized in managing product planning, parts purchasing, customer service, inventory maintenance, supplier interfacing, order tracking, etc. ERPs achieve all these tasks by the integration of multiple software modules. As can be easily understood, the landscape of complex process control is very involved. If this is combined with the lack of standard ways of integration there is a lot of effort in labor and money that goes into eliminating the friction between the modules that compose the distributed control system. This is precisely one of ICa’s goals, to provide out-of-the-box integration at all levels of the control hierarchy. 2.1.4 Sensors, Controllers, Actuators At the same time that control tends to become more complex, the computing power is becoming cheaper and cheaper. This situation is creating favorable conditions for the introduction of intelligence (controllers) close to the process, at the sensor and controller levels. Intelligent Electronic Devices (IED) are common instruments in all industries nowadays. One of the best examples of the use of IEDs is in the power substation automation industry. Thanks to IEDs, complex tasks in substation automation can be performed. In today’s substations, IEDs for protection, metering, data gathering or control purposes are a reality. Figure 5 shows the GE D25 multifunction IED [GE 03]. This IED has been specifically engineered by GE for power substation automation. The D25 multifunction IED for substation control includes a Definite Time Over Current Protection algorithm, and breaker failure protection feature. The product supports synchronizing functions, monitors voltage difference, phase angle difference, and slip frequency and supports multiple buses per feeder 1 . The unit also functions as programmable logic controller, substation local area network node, and IED gateway. The advantage of this is that the unit can replace several devices with a single unit. With the use of IEDs for electricity transmission and distribution control, substation automation projects can be managed more efficiently as the number of elements 1 This is electrical systems terminology. 28 needed to monitor the plant decreases. It must be noticed that although the complexity of the automation project decreases, the complexity of the control elements (e.g. the IEDs) has increased. For this reason, modern IEDs incorporate self-checking functions in order to verify system status. While the number of elements in the plant decreases with the use of IEDs, the software needed to manipulate the plant is more complex as the number of services provided by IEDs has grown. The advantage is that better control of the plant can be exercised as there is more information to reason upon. Figure 5: A multifunction IED from GE But among the objectives of this Thesis is not only to ease the work of developing distributed control applications for large industrial complexes but also in the area of distributed real-time embedded (DRE) systems. The focus also includes hard real-time applications. An example of this type of systems is the real-time control of automotive systems. In this case, independent autonomous subsystems must share information in hard real-time. For instance, the engine and the transmission subsystems of an automobile must share information in such a way that there is a maximum bounded latency between the input stimulus and the response. The type of sensors used in this case has stricter requirements that those commonly found in large plants. Also the resources available in the sensors, controllers or actuators are smaller (e.g. computing power, available storage space, memory footprint, etc.). Figure 6 shows a drive-by-wire box [TTTech 05] which supports communications based on the Time Triggered Protocol (TTP) [TTP 02]. This box is designed to be 29 used in automotive hard real-time applications where high power actuators are needed. The box is suitable for brake-by-wire, clutch-by-wire or steer-by-wire applications. The TTP box is also an actuator that supports direct control of a brushless DC motor. Figure 7 shows a high-performance snow groomer. This machine electronic core is based on a TTP network which controls the driving systems and the overhead winch so engine power can be easily adapted to operating conditions. The rear snow tiller, the front rake blade and the entire on-board instrumentation are deployed on the TTP network. Figure 6: A TTP drive-by-wire box from TTTech Figure 7: A snow groomer based on a TTP network (figure authoring of TTTech) In summary, the availability of better hardware resources even for embedded real-time applications has opened the path to the integration in a variety of devices of software architectures for control purposes that were unthinkable a few years ago when most control software developments were ad-hoc for every system. For instance, the use of 30 software frameworks was impeded by the scarce computing resources of deeply embedded systems. 2.1.5 Industrial Communications Protocols When talking about communications for industrial applications [Hanssen 03] there are several levels that must be addressed. Here, in this first part of the section, referral is made to large plants (e.g. petrochemical plants) and not to embedded applications. Those levels approximately correspond to the different types of networks that can be found in large distributed industrial applications. These networks are frequently denominated field bus, control and enterprise (also business or information) networks. Going bottom to top the first network is the field bus network. The field bus network mission is to link the distributed intelligence of IEDs so they can meet their tasks. Control is decentralized as much as possible so each device is able to gather the needed data from the field bus to carry out its work with local intelligence being decoupled as much as possible from a central control (control network). Field bus technology has evolved much in recent years. The problem with field bus technology is that there is not a unifying standard of field bus technology. Different field buses coexist in the market (e.g. Profibus PA [Profibus], DeviceNet [Dev], World FIP [World]). Differentiation came in one way because of specialized needs of industries like automotive, pulp and paper, process or semiconductor. Secondly, differentiation of field bus technology came because of the race for the field bus market among Japanese, European and U.S.A. companies. This situation has lead to the development of several field bus communication standards each of which is employed depending on the industry, geographical area (see Figure 8) or control system implementer’s preferences. Figure 8: Geographical distribution of field bus usage 31 Field bus networks are usually small, exchange low amount of data and have typical reaction times of about 10 ms. The second network is the control network. As happens with the field bus network there are several protocols available. Most of them are based on Ethernet. Examples of these protocols are ControlNet [ControlNet], Profinet [Profibus], Foundation Fieldbus HSE [Foundation], Modbus [Modbus] or CAN [CAN]. The requirements for this network are less stringent than those of the field bus network. They are medium sized networks which support high amount of data and typical reaction times of about 100ms. The enterprise or information network is a typical TCP/IP network. Its size can be very large, very large amounts of information are exchanged and the typical reaction time expected from such a network is about 1 s. An example of the control network layers for a continuous process plant is shown in Figure 9. ENTERPRISE NETWORK MIS Safety Business Management Data Storage CONTROL NETWORK Process Control Process Operation Process Management FIELDBUS NETWORK Field Configuration Sensing and Acting Field Management Figure 9: Process plant integration 32 Many of the communications protocols described above are not suitable for hard realtime. When critical systems come into play a more deterministic approach to communications is needed. Among the systems of this category we can cite railway signaling systems, fly-by-wire cockpits or automobile computer networks. Examples of more deterministic communication protocols are RT-Ethernet [Doyle 04], the Time Triggered Protocol (TTP) [TTP 02], FlexRay [BBE 02], and the time-predictable variant of Control Area Network (CAN) [Bosch 91], TTCAN [Har 00]. The TTP protocol approach relies on system progress based on time rather than on a system model that progresses by events. The idea is that event-triggered systems cannot easily cope with the requirements of composability, predictability, determinism and guaranteed deadlines of hard real-time systems. TTP relies in a Time Division Multiple Access (TDMA) scheme to control access to the communication medium (see Figure 10). In this protocol, access to the bus is controlled by the progression of time where the available channel bandwidth is statically divided into a number of slots which are used by the network nodes for their transmissions. The TTP uses a priori knowledge to schedule the network. This means that the access to the communication medium by each node is known at design time. Figure 10: TDMA bus access scheme in TTP TTP has several flavors; mainly TTP/A and TTP/C. TTP/A is member of the TTP family intended for low cost networks of sensors and actuators while preserving the temporal properties of TTP for field bus applications. The TTP/C protocol of the family is intended as a high speed network (25 Mbit/s) for high dependability applications. TTP/C provides fault-tolerance by the ability of tolerating one arbitrary faulty node or one faulty communications channel in the system. TTP/C can be integrated with TTP/A in an architecture where several field bus clusters have to be interconnected in a temporally predictable manner. FlexRay is based on the TTP/C protocol fundamental ideas. It is specifically designed to be introduced in the car due to the amount of electronics for automotive applications of recent years. The FlexRay protocol provides fault-tolerant time 33 synchronization via a global time base, collision-free bus access, guaranteed latency and system fault-tolerance by the use of dual communication channels. The Control Area Network (CAN) is a serial protocol that supports the prioritization of messages. CAN supports data rates up to 1 MBit. The medium access scheme used by CAN is a Carrier Sense Multiple Access / Collision Avoidance (CSMA/CA). The problem with CAN is that worst case latency can be too high. To make CAN latency predictable TTCAN can be built on top of the standard CAN protocol. TTCAN synchronizes the schedules of all the CAN nodes in the network by means of a global system time base. Among the several candidates for a real-time communication transport protocol for CORBA, a decision was made in favor of a RT-Ethernet plug-in, developed in collaboration with the Department of Automatic Control at Lund University (Sweden) and a TTP/C based transport plug-in that was developed for ICa in collaboration with the Institut für Technische Informatik (ITI) at the Technical University of Vienna (Austria) which is directed by professor Hermann Kopetz, chief architect of the Time Triggered Protocol. The TTP technology was selected because of its increasing relevance in railway, aerospace or automotive markets. For instance, Honeywell uses TTP for the digital engine control system of Lockheed Martin’s F16 fighter aircraft. Another example from the aerospace industry comes from Nord-Micro, which uses TTP as the communications protocol for the Airbus 380 cabin pressure control system. RT-Ethernet is a cheap approach to real-time applications that gives good performance for many (non safety-critical) systems on standard communications hardware. 2.1.6 Trends of Control Networks Although in the process industry always there have been the control levels of Figure 9 (with their differentiated enterprise, control and fieldbus networks), the control implementation scheme for large and small industries keeps evolving over time. Control system construction and deployment has evolved from the direct digital control, where all devices were connected separately to the control room (where the control was centralized to a single computer), to the traditional Distributed Control System implementation, where several devices are linked to a controller and there are several distributed controllers that are connected to the DCS console. In the years to come, control will be totally distributed to field devices which integrate all needed control loops. This will be the era of true distributed intelligence. Figure 11 shows such evolution of complex control system implementation where the decentralization process is shown against time. 34 PAST (Centralized) Centralized Control Distributed Control System PRESENT (DCS) FUTURE (Network Integration) Distributed Intelligence Figure 11: Trends of control system networks 2.2 Commercial State Of The Art DCSs The purpose of this section is to present an overview of commercial distributed control systems that represent the state of the art of technology. 2.2.1 Honeywell Experion PKS Experion Process Knowledge System [Honeywell 04] is a system solution from Honeywell which integrates all networks / layers of Figure 9 under a unified system from the same vendor. Experion is a single solution for the whole plant. Experion goes further than the process control a simple DCS can afford by including subsystems to 35 manage the plant at the enterprise or business level. Figure 12 shows the architecture of Experion PKS. Figure 12: Platform architecture of Experion PKS (figure authoring belongs to Honeywell) As can be seen in Figure 12, the architecture is organized in layers where four different networks are differentiated. First, there is a Local Control Network which is the fieldbus network. The process network is divided in Experion in two networks, the Supervisory Control Network and the Advanced Control Network. Finally, the enterprise or Business Network is placed atop all other layers. Experion improves former Honeywell solutions with improved process applications, plant asset management and integration of business services. For instance, Experion asset manager monitors the plant in real-time and helps to discover areas that are most likely to have a negative impact in performance or determines how well a device is performing in the context of a control loop. This kind of monitoring is useful to minimize downtime and to avoid unscheduled replacement costs. 36 Another problem in large complex plants is the operator decision system. Knowledge related to process improvement or plant safety is frequently concentrated in experienced individuals and is rarely captured and shared. In practice, this means that non-automated tasks are treated differently by different operators resulting in a loss of system overall performance. Experion PKS includes collaborative decision-support tools to let operators to prevent abnormal situations while maximizing plant production. From the business point of view, the Enterprise Network, Experion offers tools to analyze production data in real-time so performance reports can be generated continuously and not on a quarterly basis. Experion offers interfaces to pure business systems so the complete plant can be integrated from planning and scheduling to the supply chain. One of the problems of current systems is integration; Experion tries to be open by providing integration services with other non-Honeywell systems. All this integration effort always has a cost which is finally satisfied by system buyers. For instance, Experion provides support for the following field buses: FOUNDATION, HART, Profibus, DeviceNet, LON, ControlNet and Interbus. The question remains how much does this complexity for multiple communication mechanisms add to system cost. 2.2.2 Airbus 380 Cabin Pressure Control System The Airbus 380 cabin pressure control system is another example of a distributed control system that can be tackled with the ICa ORB. The Airbus 380 is the largest and most advanced airliner to date. It is a double-decker with space for 555 seats in which the seat × mile operating costs have been reduced 15-20%. The control of cabin pressure and ventilation system in such a large aircraft is somewhat complicated. First, it must be understood that the cabin pressure control system is safety critical. Additionally, the pressure in the fuselage must be comfortable for the passengers and the crew in any altitude. At the same time, it is necessary to reduce complexity in the control system and to make it as lightweight as possible. These two conditions mean that it is necessary to develop a distributed real-time system with safety critical requirements and that wiring should be as simple as possible. The wiring problem can be solved by means of a communications bus. The real-time and safety-critical problem (from the point of view of communications) can be solved by means of a suitable protocol and hardware. In the case of Airbus 380 this protocol is TTP/C due to safety requirements. 37 The control and monitoring system of cabin pressure operates autonomously controlling the air volume that flows out of the fuselage through four outflow valves. The system incorporates two safety valves and a negative-pressure relieve valve for safety. Figure 13 shows a schema of the control system with duplicated TTP communication channels for fault tolerance. Figure 13: Airbus 380 cabin pressure control system schema The control loop for this system is quite simple. The pressure controllers input variables are the cabin pressure, the outside pressure and the flight profile data (altitude and speed are important for pressure prediction). The controller generates a control signal for the required actuator to set the outflow valve angle. All communications take place by means of TTP messages. 2.2.3 AVE Power Distributed Control System AVE stands for “Alta Velocidad Española”, the high-speed railway system of Spain. The AVE railways are powered by electricity distribution substations geographically scattered along the railway path. The distributed control system of the power substations is interesting because it combines real-time, non real-time and fault tolerance requirements. This distributed control system and the power substations 38 have been implanted in Spain by the consortium formed by the Spanish company Electren and the French company ALSTOM (the AELEC consortium). Figure 14 shows a schema of the control system architecture. The main idea behind the architecture is that the power system for the railway is divided by traction substations that have several associated auto-transformation centers. In the figure, the traction substation is depicted in the upper left corner of the image inside the dotted-line square. The bottom half of the picture shows the autotransformation centers associated to the traction substation. In the upper right corner there is a remote central telecontrol post. In each of the traction and autotransformation centers there are several networks. First there is a primary network which is represented by the double rings in each of the centers and that links the center PLCs (in black). The secondary network links the PLCs of each ring with their controlled devices (the IEDs, in blue, which control circuit breakers, switches, etc. and that are not shown in the figure for simplicity purposes). Figure 14: AVE power substations distributed control system (Figure authoring belongs to AELEC) 39 In this case, those devices are mainly protection relays (fully fledged IEDs with complex functionality) which are depicted in blue color. There is a third network which links all traction and auto-transformation centers along the railway line. Fiber optics is used extensively for all networks except for some protection devices which are tripped by means of copper wire links. The primary network is based on the EFI.P protocol and is physically redundant. The protocol is deterministic and the network links all PLCs and a redundant industrial PC used as a data server and gateway to the external (the third) Ethernet network. This PC also serves as a time base for all events in the substation network by means of a GPS card. The secondary network uses two types of communication protocols with the protection relays: MODBUS and IEC-870-5-103. The first is a fieldbus protocol for serial communications. The second is a specific protocol for communication between Remote Terminal Units (PLCs) and protection devices. The third network uses Ethernet and is the operator applications network. All control applications on this network use OLE for Process Control (OPC) to communicate with the data server which acts as a gateway between the second and third level networks. Finally, there is a fourth network that uses the protocol IEC-803-5-101 to communicate with the remote telecontrol post. There is an OPC / IEC-803-5-101 gateway to provide the conversion between protocols (middle of the upper half in Figure 14). The SCADA of this control system is based on a General Electric SCADA development tool called CIMPLICITY and its runtime engine CIMVIEW. It also uses at least six different communication protocols which make integration of the different parts of the system complicated and, consequently, raise costs and risks. Among the integration problems posed by this system there are some interlocking operations which must coordinate elements of the traction and auto-transformation substations. For this, there are some automatisms running in a control PC connected to the Ethernet / OPC network running on the ISAGRAPH software. This system structure corresponds to the present state of distributed control systems of Figure 11 as is the state-of-the-art solution of section 2.2.1. The difference between the two is that the AVE system is a real example of the integration of hardware and software from multiple vendors and the problems (and advantages such as avoidance of vendor lock-in) this implies. 40 2.3 Critique Statement The three reviewed systems show some examples of current state-of-the-art distributed control systems. First, a whole solution from a world-class manufacturer is shown. This solution addresses process control of large plants where the promise of seamless integration of the hierarchy of control layers is delivered. The main problem with this solution is that vendor lock-in is very difficult to avoid. In some cases vendor lock-in does not need to be bad but, common sense indicates it is better to have a wide variety of suppliers to buy products from. Vendor lock-in eliminates competitors and that almost always results in an imperfect market. Although this kind of systems integrates with other systems and protocols, it always tries to promote their own, so the benefit for the DCS manufacturer can be maximized. The problem here is that proprietary technology is imposed to system proprietors. Interoperability must be independent of DCS vendors. ICa can help solve this problem by means of the public CORBA specification of interoperability. The second system shown, the Airbus 380, is completely different. It is a distributed system with hard real-time, fault tolerance and safety-critical requirements that is developed mostly ad-hoc. In this case, the choice of a suitable communications protocol is less wide because of the stringent system requirements. However, the communication system to be employed (TTP/C) lacks the tools to ease the development which results in higher costs in an area of control in which development costs already result very expensive. Also, there are not suitable frameworks for the development of real-time applications. For instance, tools to control processor, memory or communications resources. ICa assists in solving this problem by providing this kind of control. The third system, the AVE power substation control system, is a large system geographically scattered which takes the multi-vendor integration approach and in which different types of requirements for the different types of networks in the system must be met (from real-time to non real-time but always fault-tolerant). In this case the problem is twofold. First, there is the integration issue of products from multiple vendors. There is no consensus on hardware platforms, operating systems or communication protocols. Second, there are different types of requirements for the different parts of the system. ICa solves these problems by a) adopting an interoperability protocol that can be adopted by all manufacturers, b) providing ORB releases with real-time, non real-time and embedded characteristics. 41 Chapter 3 Distributed Component Middleware 43 3.1 Definition of Middleware Middleware can be accounted for as the glue that links together all the elements of a distributed system. It is the piece of software that mediates between applications and the network and that isolates the application from many of the lower level details of the operating system. Middleware aids in carrying out the interaction between components across a heterogeneous distributed system. There are several types of middleware (see Figure 15) but the one with most wide acceptance is the Object Request Broker (ORB) middleware. This is the result of its object orientation, providing the advantages offered by objects to the distributed world. The objective of middleware is to release developers from the common, complex, lowlevel or mundane tasks of module interaction in a distributed system. This is achieved by implementing a common abstraction for the programming of distributed systems. All modules of the distributed system must conform to the abstraction in order to be able communicate. Thus, middleware is used to make processes residing in one or more host computers to interact. The middleware provides to applications a set of services of coarser granularity than those of the operating system in the form of Application Programming Interfaces (API) that enable them with the following properties among others. • • • • • To locate services across the network in a transparent manner. This means that the distribution is transparent for applications. For the component parts of the systems it is as if the system were monolithic. To be independent from network services. The applications are not aware of the network; the middleware is in charge of managing the intricacies of the network access details. To be reliable and available. Programming the network is a complex task and the middleware provides a proven and reliable way to glue together the components of a distributed application. To scale up in capacity without losing function. Middleware provides services to control the lifecycle of the components of the distributed system and to control the resources available for the whole system. To resolve the issue of heterogeneity, allowing the construction of systems where there is heterogeneity of hardware, operating systems, network protocols and programming languages. 45 Application/ Industry/Business aware Applications Vertical Domain Services and Frameworks (Telecomm, Groupware, Utilities, etc.) Distributed Object Services (CORBA, DCOM, ActiveX) Client / Server Services (RPC, MOM, TPM) Primitive Services (Terminal emulation, File transfer, email) Network Programming Services (Sockets, LU6.2) System aware Network Services (TCP/IP, SPX/IPX, ATM,SNA) Figure 15: Types of middleware services 3.2 Taxonomy of Middleware Middleware can be classified according to the four following forms: • • • • Transaction Processing Monitor Middleware Remote Procedure Call Middleware Message-Oriented Middleware Object-Oriented Middleware. The classification distinguishes the different types of middleware according solely to the different types of programming machinery involved in the communication between parts of the distributed system. Essentially, the criterion is based in that all types of middleware define a uniform scheme to access other components of the distributed system scattered across the network. 3.2.1 Transaction-Processing Monitor (TPM) Middleware TPM is a client / server middleware specifically designed for transaction applications. This is a mature software technology that is employed in applications involving data management, database access, order delivery, etc. The TPM middleware is designed to provide access to thousands of clients in a distributed client / server environment by 46 controlling and mapping client transaction requests onto a set of service routines that multiplex the transactions (see Figure 16). The service routines are critical to the architecture of the system and are the only client-routines that the server system interacts with, avoiding the overhead of dealing with thousands of clients. The advantage of this technology is that the server system only sees a reduced number of interactions through a small set of routines and not the whole amount of clients. Client access is prioritized according to the type of service requested. Also, the logic from the business rules of transaction processing can usually be moved from the clients to the transaction processing monitor making it easier for administrators to deploy system upgrades. TPM technology scales well when the number of clients grows as all requests are multiplexed by the TPM. When the number of clients is too large, it is possible to add more TPMs to better handle multiplexing of requests. The TPM market is dominated by a few vendors. They provide large systems that integrate with a great variety of business processes and offer ways to access the functionality of the TPM in several ways. For instance, Tuxedo TPM from BEA has a CORBA and an Application-ToTransaction Monitor Interface (ATMI) interface to access the TPM infrastructure. client client client client Service Routines client Transaction Processing Monitor client Server client client client client client Figure 16: Transaction Processing Monitor Most of these TPM products offer advanced features as load balancing, management features or fault tolerance features. Some of the most widely employed TPMs are: 47 • IBM CICS: CICS stands for Customer Information Control System. The name stands for a family of application servers and on-line transaction management infrastructure for mission-critical applications. CICS has interfaces available for COBOL, C, C++, Java or PL/I. • BEA Tuxedo: The product includes an application server, a TPM, a backend transaction processing infrastructure and two application programming interfaces one for CORBA in the C++ language and the ATMI. • HP Encina: Encina uses the Distributed Computing Environment as an RPC basic infrastructure over which the transaction processing monitor is built. HP has enhanced the RPC mechanism to give it transactional semantics defining what they call Transactional RPC. 3.2.2 Remote Procedure Call (RPC) The RPC is a client / server infrastructure that allows developers to access components of a distributed system by means of function calls. In the case of RPC there is not a defined software layer that implements the middleware, middleware is embedded in the client and server applications. To embed the Remote Procedure Calls, at compile time stubs are created for the client and server sides which are compiled and linked with the rest of the application. The stubs are invoked when the client program calls a function which must be serviced by the remote RPC server program. This mechanism is depicted in Figure 17. RPCs are synchronous calls between clients and servers although some vendors offer asynchronous functionality. When a remote function is invoked, the calling procedure (the client) must wait for the server to end the processing of the function before control is returned to the client application. Notice that the function call blocks until the server application replies. This mechanism can be chained if the server is at the same time client of another application and invokes another remote server when servicing the original RPC. The advantage of the RPC mechanism is that it masks the function calls as if they were being made locally although the inner service workings behind the masked function call are rather complex. Function arguments must be marshaled and a message to the remote RPC service where the function is implemented must be sent and the reply returned. This is generally done synchronously so a tight coupling between the client and server programs exists, being it possible for a client to remain blocked for a long time in a lengthy procedure call. To keep the system running with this kind of infrastructure the only solution is concurrent programming, multithreading the system which in turn increases complexity. 48 Server Transport Network Transport Client RPC Stub Network The major drawback of RPC is that it is a relatively low level mechanism. The details of the underlying communications mechanism are very little hidden from the developer. RPCs provide support for a procedural approach to system development and, while in some cases this might be good there are myriads of situations in which an object or component-based approach is significantly better. Being the communication model synchronous increases the complexity of development in the case lengthy remote procedure calls must be done. RPC Stub Application specific RPC Figure 17: Remote Procedure Call Middleware There is a variety of vendors in the market which provide RPC software with both synchronous and asynchronous functionality, among them there are the following products: • • • Entegrity Solutions Gradient RPC. Entegrity is a provider of business to business solutions based on the DCE RPC. Gradient RPC difference from previous RPC products is that the communications employed for RPC are secure as demanded by business to business communications. Compaq’s (HP) DCE. Compaq sells RPC products based on the Open Group’s DCE for the OpenVMS and True64 UNIX operating systems. Sun’s RPC. As the company is nowadays oriented towards the Java language, it currently offers RPC products which expose a Java API for XML-based RPCs (JAX-RPC). XML-based RPC are remote procedure calls specifically designed to be invoked over the internet. For this reason, the transport protocol used is HTTP and the encoding is XML. Hence its name, XML-RPC. The purpose of this product is the interoperability of web services among different platforms and languages. 49 3.2.3 Message Oriented Middleware (MOM) The MOM is a client / server architecture that supports asynchronous invocations between clients and servers. In a MOM, software architecture messages are not directly sent to the server, there is a message queue at the client and server sides (see Figure 18). The purpose of the queue is to store messages in case the receiving application or process is busy performing another task. Once the sender has released the message, it can continue its processing. The queue that stores the message will try to send it to the recipient process until the message is accepted. Once a reply is built by the recipient, it will be sent to the queue which will try to send it to the original sender of the message until it is again accepted. The advantage of MOMs resides in that it decouples clients from servers by means of the queues. This architecture allows MOM-based applications to process huge amounts of messages as they are stored in the queues and that will be processed when there are computing resources available. Most MOMs will accept that more than one process reads messages from a queue, being the system able to adapt message processing to the number of messages held in the queue. Also, most implementations support synchronous message passing. MOMs are well suited for object-oriented systems and for event-driven applications. For object-oriented systems, the message passing implemented by MOMs fits well into the peer-to-peer communications model of objects. In the case of event-driven systems, messages can be passed to the MOM infrastructure when the event occurs. The responsibility of forwarding the message to the server is then on the middleware and the client can continue its processing. As the basic communication process is non blocking (messages are stored in the middleware), there is a potential danger of overloading the network by issuing thousands of messages to a server which is blocked or simply that can not follow the message arrival rate. This situation is corrected in most MOMs by defaulting to synchronous message processing when such a situation is detected. Unlike, RPCs which always block (although, as mentioned, there are vendors that provide asynchronous implementations), MOMs provide a more flexible approach to systems development. Notice, that MOMs are especially good for applications whose modules cooperate with a certain degree of independence and where the server may be unavailable at the moment the message was issued. For this reason, it is well appreciated technology for in Enterprise Application Integration (EAI). 50 Server MOM With message queue Transport Network Network Transport MOM With message queue Client Application specific proprietary messages Figure 18: MOM Architecture Among the vendors that dominate the market there are the following: • • • 3.2.4 IBM MQSeries: This MOM has been designed to integrate heterogeneous environments and supports more than 35 different platforms. The MOM offers an API called Message Queue Interface (MQI) which must be used by all applications that want to communicate with other systems. The MQSeries supports standards such as XML or the Java Messaging Service (JMS). Software AG EntireX: EntireX is a component-based application integration product based on a message server brokering software. This MOM handles security information and supports synchronous and asynchronous communication models. TIBCO Enterprise Message Service, SmartSockets and Rendezvous: TIBCO manufactures software for messaging and multicast products. EMS is a message queuing product that supports the asynchronous communication model, message persistence and transactional communications. SmartSockets is a product that supports a publish / subscribe model in which messages are delivered only to interested applications. There is a version of the product SSLSmartSockets that provides secure connections among applications. Rendezvous is designed to increase the scalability for the overall enterprise infrastructure by providing message routing and filtering services. Object-Oriented and Component Middleware There is a distinction between component and object-oriented middleware and the reminder types of middleware. Object and component middleware key-concept is that 51 of the interface. Every service in this middleware is accessed through interfaces and no implementation details of the service need to be known by the requesting applications. However, this is also a concept that is established by the very object concept. There are some other reasons that back the promotion of this type of middleware: • • • Most programming languages do not support distributed system construction, The object or component model alone does not support distribution across machine boundaries, Large systems are constructed in multiple programming languages, operating systems and hardware. They are based upon heterogeneous environments. The reasons above are what lead to what are called “stovepipe” systems. Systems which are developed ad-hoc, with proprietary integration solutions. Instead of this, the object-oriented middleware defines the requirements to build systems where there is a standard way for internetworking and where there are standard interfaces to access common services of the infrastructure. In the object-oriented middleware, there is support for the object-oriented programming model and thus the same benefits that object systems have can be applied to this type of middleware. The most important one is that object systems map well the entities from the real world. Then objectoriented middleware maps well the entities of the real-world that are distributed in nature. Figure 19 shows a general architecture for object-oriented middleware. local object A proxy object B OOM object request broker / object manager OOM object request broker / object manager remote skeleton object B object B Figure 19: General architecture for object-oriented middleware Object-oriented middleware uses an Object Oriented Manager (OOM) or an Object Request Broker (ORB) as the mediator software layer between local and remote objects. Remote objects expose their interfaces which can be accessed as if they were local objects. For this, the remote object interface is accessed locally via a proxy 52 object. The proxy object acts on behalf of the remote object and its responsibility is to forward the request invocation to the middleware layer, the Object Request Broker that can be thought of as a software bus, which will take the necessary actions to correctly deliver the request. The basic mechanism used for communication is that of remote method invocation. Remote objects can be uniquely identified by means of their object references. Object-oriented middleware has several differences with other types of middleware: • • • • • • • There are neither clients nor servers. In object-oriented middleware there are only objects that provide services and that receive services from other objects. Location transparency. The use of proxies decouples the service user from the service provider. When implemented, this yields the advantage that the location of the service can be done independently of the service user. Design of service distribution can be postponed. Services are provided by objects and as location transparency is one of the middleware properties, users of services are unaware of where the computing resources for the service are located. The system can be developed locally and distribution can be delayed until system testing. The system is perceived as a whole by the objects, as if it were a monolithic system. Migration of objects. Objects in the system can be moved dynamically without affecting the users of the services they provide. This is achieved by common services built in the middleware layer. For instance, in the case of CORBA the ORB is able to issue “locate forward” messages to resolve a new object reference in case an object is not found in its previous location. Concept of services. Object-oriented middleware introduces the concept of common services. These are services that can be provided by distributed objects in a standard way (e.g. vendors can bundle the services with their products). Examples of these services are the naming service, event service, object lifecycle service, etc. The system architecture is open and scalable. Distributed object systems are able to scale without modifying the underlying system structure or architecture. The system simply grows up by increasing its number of objects. Independence of programming languages and hardware platforms. The key concept is interface. Distributed Object Middleware uses Interface Definition Language (IDL) to define object interfaces. The interface concept is useful because it allows defining a common object model that is expressed in IDL. IDL is independent of programming languages or implementation details. At the programming level, IDL is mapped to any number of programming languages by IDL compilers making the system independent of 53 languages or particular hardware implementations. The specific language code generated from IDL files contains the proxy code to be used locally to access remote objects (stubs) and the templates of remote object services to be implemented (skeletons). There are three different approaches to object-oriented middleware that dominate the market, OMG’s CORBA, Sun’s Java/RMI and Microsoft’s DCOM. Each of them has some advantages and disadvantages that will be presented in the following sections. 3.3 Components vs. Distributed Objects There is not a clean distinction between components, in a general sense, and distributed objects. In its definition a component shares two basic properties of objects; it has interfaces and provides data-hiding. A component is a group of functions and related data that can be encapsulated into one or more objects that form the component. The functions can be accessed by a running program residing in the same or in another host. For instance, a cryptographic component groups functions to provide different cryptographic algorithms to its clients. The component can hold several objects, each one being responsible of a different cryptographic algorithm. In some cases, the component will have only an object, making then difficult the distinction between distributed object and component. In the common case, the component is constituted by a set of collaborating objects. In the case of legacy systems, not an unusual case, a component can be formed by a module of a system that has been wrapped with CORBA interfaces. Thus, for the scope of this work, the description of component accepted agrees with that of [Brown 97], “A component is an independent set of deliverable services”. A fundamental difference between objects and components is that in the essence of objects is the representation of some mental or real-world form of the envisaged system. In the essence of components this idea does not exist. The idea of componentry is to build software out of components. Software must be componentized so it can be built from components. Software components are a way to encapsulate functionality, often in the form of objects, upon a software infrastructure so the component can live in isolation and autonomously from the rest of the system. Software components have been built upon the ideas of software objects, architecture, frameworks and patterns. It is useful to think of a component as a system module which is linked to middleware. The transformation of modules into components of a middleware system reduces 54 overall complexity as the modules are easily reachable from anywhere in the system without making them tightly coupled. Different components of the systems can be changed or modified independently of the others. Components can be classified according to many different criteria (e.g. according to component stakeholders, developers, integrators, clients). In order to stress the difference between objects and components Figure 20 shows a classification of components and middleware according to their granularity and to their scope. Advanced Component Models Basic Component Models Infrastructure Components OLE CORBA JavaBeans Business Components Domain Components GUI Component GUI Component Business Component Process Component COM+ Enterprise JavaBeans CORBA Component Model Utility Component (Database, Transaction) Business Component Utility Component Smaller, less complex components Business Component Larger, more complex components Figure 20: Classification and types of components 3.4 Open Group’s Environment Distributed Computing The Open Group is an international vendor and technology-neutral consortium. It was formed by merging the Open Software Foundation (OSF) and X/Open Ltd. Its mission is to help developers, vendors and clients accelerate the development of open systems. The consortium now has more than 300 members that along with customers unify their vision and improve consensus-building on how open systems should be implemented. The Distributed Computing Environment (DCE) is a piece of software technology to build distributed systems in heterogeneous environments. DCE was historically the 55 first multi-vendor middleware standard technology. Over the years, DCE has become a basic technology upon which other middleware for distributed systems has been built (e.g. transaction monitors). DCE comprises a set of software services residing on top of the operating system. It is middleware that uses low-level operating system and network primitives to carry out its tasks. As it was promoted by the Open Group, it is vendor neutral and has been designed for and runs on multiple platforms. The Open Group provides DCE in source code form to be bundled by vendors with their products. DCE usually ships with the operating systems and is available almost on every platform. The DCE services include, • • • • • • Remote Procedure Call (RPC). For client-server communication, so distributed resources can be accessed across a network. Security Service. This service provides user identity authentication, authorization to distributed resource access by users and account management for users and servers. Directory Service. The directory service provides a naming model across the distributed system. Time Service. To provide network-wide clock synchronization. Threads Service. The threads service is designed to help servers accept concurrent requests increasing the execution capability of the system. Distributed File Service. Provides access to files across a network. When linked to a directory service, common names can be used in the whole system to address networked resources. DCE was not thought of as an object-oriented technology. As the use of objects is increasing in all types of software, there is a significant challenge in what will the evolution of DCE be. DCE was initially written in C language (not an OO language) and its only standard specification is written then in that language. There was a first attempt, with the release of the DCE 1.2.1 standard in 1996, to specify an interface for C++ but, in practice, the support from vendors for the DCE C++ interface has been partial and fragmented. This lack of support severely hampers object-oriented development using DCE. The evolution of DCE must go through the world of objects where it will encounter severe competition by CORBA and DCOM. DCE is now a mature product and although it is largely used, mostly in legacy systems, it seems clear it will not play a significant role in new developments. Among its advantages, DCE has tested support for interoperability in heterogeneous environments and mature services that increase systems reliability. However, there are also great disadvantages for recommending its use: 56 • • • • There is not practical support for object-oriented languages. Weak market position in relation to CORBA and DCOM. DCE provides rather low-level services. There is no component market for DCE. When looking for a middleware technology for ICa, some years ago, DCE was the initial election. At the end, it was replaced by CORBA, then emerging, for the above reason concerning object-orientation that was deemed critical for the ICa infrastructure. 3.5 OMG’s Distributed CORBA Architecture The Object Management Group is the largest software consortium in the world. It has more than 500 members all of them related in one way or another to the computer industry. The OMG work began back in 1989 founded by just eleven companies. It is committed to the development of vendor-independent technical specifications for the software industry. Basically, its primary mission comprehends the creation of a marketplace for component software and systems by the establishment of industry guidelines and object management specifications. The Common Object Request Broker Architecture (CORBA) is it most famous specification, together with UML, that deals with distributed object-oriented middleware. CORBA instances the interfaces for the entities defined in the Object Management Architecture (OMA), the OMG’s reference architecture for distributed object systems. CORBA is already a mature technology, created before DCOM and after DCE. It has always been object-oriented and vendor, language and platform independent. CORBA’s objective is that of the OMA, to enhance interoperability and integration among distributed systems. As other types of middleware, CORBA uses an Interface Definition Language for the specification of objects interfaces. In the case of OMG or CORBA IDL, this language has been standardized as an ISO standard (ISO / IEC DIS 14750). The CORBA specification will be dealt with in depth in the following chapters. Nevertheless, it is worth to enumerate some of its advantages in this section. • • The CORBA specification is open. The specification follows an open process for its evolution or for the creation of new specifications. CORBA is object-oriented from inception. However, it can also be used to encapsulate non-object-oriented systems as it defines mappings for a huge 57 • • • • • • amount of languages. This means that CORBA is able to integrate well with legacy applications and systems. CORBA has support for an overwhelming number of different platforms and operating systems, being the heterogeneous environments its natural habitat. CORBA is supported by hundreds of vendors. CORBA and the OMA provide support for the concept of services, facilities and for domain specification which improve system development and reliability by the use of standard middleware components. The ORB concept provides transparency to the distributed system in several ways. Location transparency, access transparency, concurrency transparency, migration transparency, etc. CORBA emphasizes dynamic interface availability when needed. CORBA provides a Dynamic Invocation Interface (client side) and a Dynamic Skeleton Interface (server side) that confer greater component mobility than other types of middleware. CORBA is easier to learn and has a cleaner object model than DCE, Java / RMI and DCOM. DCE, apart from not being object-oriented, suffers from being too close to the operating system and developers are worried about details that have nothing to do with the application functions. On the other hand, DCOM is based on complex C++ virtual method calls, which yields complex interfaces usually generated by specialized tools. This makes complex to understand large systems. Regarding the last item in the previous list, CORBA has always been criticized as a difficult to learn technology. As happens with most things, CORBA can be a difficult or easy to learn technology if compared to other options. Easy or difficult is not an absolute magnitude neither is it something that can be measured objectively. Regarding DCE, Java / RMI and DCOM, CORBA is easier to learn than the other three. As an example take the following IDL excerpt of an interface for the three object-oriented types of middleware. The IDL code specifies a sensor interface with an operation to read its value. DCOM IDL CORBA IDL Java/RMI IDL [ uuid(7371a240-2e51-11d0b4c1-444547850000), version(1.0) ] library Sensors { importlib("stdole32.tlb"); [ uuid(BC4C0AB0-5A45-11d2- module Sensors { interface ISensor { float get_ value ( in string varName ); }; }; package import import 58 Sensors; java.rmi.*; java.util.*; public interface Sensor extends java.rmi.Remote { float get_value( String varName ) throws RemoteException; 99C5-00A76814C655), dual ] interface ISensor : IDispatch { HRESULT get_value( [in] BSTR varName, [out, retval] float * rtn); } } [ uuid(BC4C0AB3-5A45-11d299C5-00A02374C655), ] coclass Sensor { interface ISensor; }; }; File: SensorLib.idl File: Sensor.idl File. Sensor.java 3.6 Microsoft’s Distributed Component Architecture The Distributed COM (DCOM) extends the functionality of the Component Object Model (COM) to support inter-host communications. In COM, the interaction between clients and components is defined without the need of mediating middleware or any other system component. But, as in the operating system processes run in virtual address spaces, it is not possible to directly communicate with a component of a different address space. For this reason, COM uses an inter-process communication mechanism (LPC or Local Produre Call) to allow clients access the components out of its address spaces (see Figure 21). Client COM Run-time Security Provider DCE RPC COM Run-time Security Provider LPC Component DCE RPC LPC Figure 21: COM components running in intra-host processes As can be seen in the figure, COM relies on the DCE RPC mechanism to forward requests between clients and servers. In the case where client and component reside in different hosts, the previous mechanism is not valid. The obvious solution has been to 59 replace the local inter-process communications with a network protocol that allows COM components to be distributed over a network. As in other types of middleware this is done without any of the involved endpoints noticing that they are placed in different locations. The DCE RPC and the security provider layers of COM are used to generate network packets that conform to the DCOM wire-protocol specification (see Figure 22). Client COM Run-time Security Provider COM Run-time Security Provider DCE RPC Protocol Stack Component DCE RPC Protocol Stack DCOM networkprotocol Figure 22: DCOM inter-host architecture The figures show a simplified architecture, as the RPC must be object-oriented an additional layer among the COM run-time and the remote procedure call is used to match objects and RPCs and to resolve object identities. From this point of view and from that of the DCOM architecture, it is more an application of the DCE technology than a truly new approach to distributed object computing. As COM is object-oriented but DCE is not (although it has some extensions for it), the mix results in complex proprietary interfaces to distributed systems. This is one of the reasons why tools are necessarily employed to generate DCOM interfaces. In platforms (UNIX and OS/390) where DCOM has been ported but where there is not good support for development tools, the problem of developing interfaces becomes a hard to solve one. As with other types of middleware, DCOM has some advantages and disadvantages. • • DCOM has a large user base and component market. DCOM is well suited for desktop applications but not for enterprise applications. 60 • • • 3.6.1 Middleware is free in Microsoft Windows platforms. It comes bundled with the operating system but outside the Microsoft world there is minimal availability for DCOM. The programming style of DCOM is complex. The interface specification is complex as it is a revamp of earlier technologies such as OLE. Deciding on DCOM is deciding on Microsoft. DCOM, with a few exceptions, only runs over the MS Windows operating systems and supported hardware platforms. There is no interoperability in heterogeneous environments. .NET Framework In the rapidly changing world of Microsoft nothing is built to last. After DCOM was built upon OLE and COM, Microsoft kept on producing new developments upon the previous infrastructure. The first of these developments was MTS, the Microsoft Transaction Server which embeds the functionality of a transaction monitor and that of an ORB for components. The name of MTS is badly chosen because it is more than a transaction processing monitor. After MTS, they announced COM+. A good definition of COM+ is the following. COM+ = COM + DCOM + MTS + MMQS + Services. [Pascual 00] MMQS stands for the Microsoft Message Queue Server. The additional services provided by COM+ include load balancing between component servers and a new event model. COM+ is an extension of the functionality of DCOM and MTS that is identified with Microsoft’s DNA; the Distributed interNet application Architecture that is a methodology for building distributed applications. DNA was the forerunner of .NET; the last initiative from Microsoft that introduces a completely new object model based on the eXtensible Markup Language. All this has occurred in the last 6 years. Certainly, IT managers of process plants would feel their nerves crisping if they follow and implement all Microsoft’s initiatives. In the industry, systems are expected to have a long lifespan and to evolve with an integrative approach. The architecture for .NET is a multi-tier (3 tier) client / server model where applications are built by composition of web services that work together to provide the application’s functionality. Again, everything is based upon the COM and DCOM infrastructure but with a great difference. In the .NET model web services are black boxes that can be accessed only via HTTP and XML. Web services can not be accessed via IIOP, DCOM or RMI. This point clearly makes the .NET architecture not usable in the industrial environment where there are strict requirements regarding 61 performance (at least the web services concept). In .NET messages are represented in XML and are sent to web services in SOAP format (Simple Object Access Protocol) via HTTP. 3.6.2 OLE for Process Control OLE for Process Control or OPC for short, its backed by the OPC Foundation which is supported by Microsoft. The foundation has more than 300 members of the automation industry. The OPC Foundation objective is creating and maintaining open specifications (sic) that standardize the communication of acquired process data, alarm and event records, historical data and batch data to multi-vendor enterprise systems and between production devices (PLCs, RTUs, DCSs, etc.). OPC was originally based on the OLE, COM and DCOM specifications and defines a standard set of objects, interfaces and methods for use in process control applications. The first OPC specification tried to address the problem of data access and now is called the Data Access Specification. As of today, OPC counts with many more specifications regarding security, alarms and events, historical data, etc. The problem that OPC tried to initially solve is what is called the I/O Driver problem (see Figure 23). If each software driver presents a different interface to applications, then the application needs a specific piece of code to access data from each driver. Display Application Software Driver Trend Application Software Driver Software Driver Report Application Software Driver Figure 23: The I / O driver data access problem 62 The OPC foundation solved the problem by creating a software bus. If all the involved applications (OPC clients) and device drivers (OPC servers) speak OPC a software bus is created and data can be accessed uniformly across the process control application. This means all blue, red, green and gray rectangles of the figure become of the same color, needing then only a single piece of code (OPC) to communicate with all devices. As the main drawback of the technology, OPC is built upon COM and suffers from the same advantages and disadvantages of it with one difference. OPC can run in more heterogeneous hardware (at the device level) than its COM counterpart. This is possible, due to the interest of device vendors in providing OPC servers with their products. As with other technologies from Microsoft, OPC and COM have not been initially thought as part of a real-time system. This means that OPC has no support for realtime features and lacks the concept of deadline or priority in the worst of cases. It is true that it supports time-stamping but there are no guarantees as to when the timestamped value will be obtained. From the OPC client-side point view and for some process control applications, Microsoft Windows is not a real-time operating system so COM and OPC who are sitting over the OS cannot also offer real-time capabilities. The final result is simply that OPC cannot cope with all the demands of present process control systems. Without disregarding these considerations, the OPC technology proves useful in those applications where the overhead introduced by OPC is less than the time constants of the controlled systems, typically these are systems with large inertia like steam boilers and akin processes. Whereas OPC is a useful technology for some kind of processes it does not cover the full range of requirements present in control applications. In the next chapters, the way in which CORBA-based technology can address all these requirements will be demonstrated. 3.6.3 Microsoft’s View of DCOM and CORBA Microsoft recognizes in its own documents that CORBA is a competing specification for distributed object computing. Microsoft, knowing one of CORBA’s weakest points, criticizes its lack of a binary format specification as one of the reasons why there is not a market for off-the-shelf reusable components. In words of Microsoft, this means that component providers have to supply source code for their components or compile and test not only for each target platform but also for each target ORB implementation. 63 Although, the claims of Microsoft are true they do not hide the fact that Microsoft suffers this very same problem. It is true they have a binary format, but it is their own binary format that only runs on one single platform. No consensus is reached with any other manufacturers and, as our premises include that we are addressing heterogeneous systems in nature, it results that DCOM simply can not interoperate in this kind of environment (further Microsoft Windows only runs on Intel hardware 2 ). On the other hand, much has been gained from former CORBA implementations in what refers to compliance to the specification and libraries of components can be provided in object code form for each platform. The second claim from Microsoft is related to broker interoperability. They claim that CORBA components will only be able to run over a predetermined ORB. Microsoft defends the position that ORB specific and proprietary extensions sacrifice cross-ORB interoperability and thus the ability to develop components for several ORBs with a moderate effort. Although it is also true that most implementations provide extensions in the form of helpful gadgets (most in the form of language macros for redundant tasks), these extensions do not impede interoperability, one of CORBA major achievements. In the author’s experience developing CORBA components, it has been possible to run exactly the same software implementations over the following ORBs: Borland’s Visibroker, Washington University’s TAO, IONA ORBIX, Vertel’s e*ORB (now bought by PrismTech), OIS Orbacus and SCILabs’ ICa. The only task needed for this level of interoperability consists in recompiling the component IDL files for each ORB implementation to generate the object’s stubs and skeleton whose inner structure is not standardized by the CORBA specification. This is certainly much more regarding component interoperability than what Microsoft ever offered. 3.7 Sun’s Java Distributed Component Architecture Java / RMI is the basic infrastructure for Sun’s distributed component architecture. Java is an interpreted, object-oriented language designed in its origin to be used in distributed applications and to be portable across platforms. For portability, Java relies on the concept of virtual machine. All Java software runs, theoretically, over the same software platform (the Java Virtual Machine) that sits on top of the operating system. Distribution in Java is achieved by means of Remote Method Invocation or RMI that An exception is Microsoft Windows Embedded (CE and XP Embedded) which supports other CPU architectures (e.g. ARM, MIPS, SH4, etc.) from different manufacturers. 2 64 allows objects residing in one virtual machine to access objects residing in another virtual machine. RMI, the invocation mechanism in Java, is a client / server model for distributed applications. Since its inception one of the central or unique features of RMI has been its ability to download the bytecodes of an object’s class if the class is not defined in the receiver’s virtual machine. This means that new types can be introduced in a remote virtual machine being it possible to change the behavior of an application dynamically. The RMI architecture consists of three different layers (see Figure 24). • • • The Stubs and Skeletons layer. This layer implements the client-side stubs and server-side skeletons needed for communications. Basically, the functionality of this layer comprehends the marshalling an un-marshalling of method arguments and the notification to the Remote Reference Layer that a new request is ready. The behavior upon reply return from the server object is reverse to that explained. The Remote Reference Layer. This layer implements the remote reference protocol independently of stubs and skeletons (e.g. invocation to a single object or to replicated objects) and transmits the request to the transport layer in a stream-oriented connection abstraction. The Transport Layer. The transport layer is in charge of the setting up and management of connections and to carry out object tracking. As in CORBA, clients making use of remote objects need a proxy or stub that acts on behalf of the remote objects. The object is invoked as if it were a local object. Application RMI System Client Server Stubs Skeleton Remote Reference layer Transport Figure 24: RMI architecture It is worth noticing that the RMI only interoperates among objects written in Java language so it is language dependent. In fact, in the Sun’s component model represented by the JavaBeans (an extension to Java objects that is a first component model designed for GUI and desktop objects) and by its extension EJB (the newer 65 component model Enterprise JavaBeans) reliance is put on IIOP if communication is needed with objects written in other languages. For this, the OMG defined a Java-toIDL binding that enables direct interaction with an ORB. JavaBeans are utilities that are client-side bound. In the component model of Sun there was need also to address the server-side with components that support features like load balancing or resource pooling. This was a must as distributed applications based in the component model need to scale well and Sun produced the EJB specification to address these issues. EJB was initially a specification not an implementation and Sun relied on third parties to provide implementations. Initially, Sun named the Java Class Libraries and the Java Virtual Machine (JVM) as the Java Development Kit (JDK). In 1999, the JDK was extended and it was renamed after Java 2 Platform. There are different versions of the Java 2 Platform depending on whether the applications are for clients, servers or mobile devices. The most popular Java platform has become J2EE, the Java 2 Platform Enterprise Edition (see Figure 25). Application Programming Model EJB Java Server Page Transactions Messaging Mail Servlets Applets Container Connectors JavaBeans Tools CORBA RMI Database Naming / Directory Figure 25: The J2EE architecture Among the disadvantages of Java concerning the development of distributed control systems, there are the following among others. 66 • • It is very slow compared to CORBA. This is true if compared to other programming languages, C++ for example. The statement also holds when compared to other types of middleware (RMI has a poor performance compared to CORBA). Java cannot directly access operating systems and hardware functions. It runs on its own private virtual machine so non-Java access to lower layers of the virtual machine needs to be coded in the Java Native Interface (JNI) 3 . 3.8 Comparing Apples to Apples Throughout this chapter, we have discussed about the taxonomy or classification of the different types of middleware available in the market. Also, there has been made a distinction between objects or distributed objects and components and it has been mentioned that almost all types of middleware support a certain component model. Even there are cases as that of Sun where more that one component model is used. Table 2 summarizes these concepts. Software Properties Distributed Middleware Object Model Component Model Advanced Component Model Data Access Messaging Naming and Directory OMG Sun Microsoft CORBA RMI, CORBA DCOM CORBA CCM CCM Java JavaBeans EJB COM COM, COM+ MTS (COM+) Not available JDBC Event, Messaging specification Naming Service Trading Service JMS ADO, OLE DB, ODBC Event Server, MSQM Active Directory JNDI Table 2: Classification of middleware features In the table above, the different overall abilities of each type of software are listed. One could be tempted to compare the EJB with the CCM component models but it An exception is that real-time versions of Java allow to interact with physical memory. 3 67 would render invalid results unless trying to reach very general conclusions as the target focus of each component model is pointed towards different uses. Simply put, an apple (component model) to apple (component model) can not be simply done. In the scope of this work, we are worried about industrial distributed control systems based on a common middleware technology no matter the layer of control where the middleware should work. Enterprise-oriented component models (EJB) and COM+ are less interesting for us than the OMG’s one as the properties of the enterprise we are considering are different from those of the former models. In some of the control layers, components could be used but in others only distributed objects can be employed. The main reason for this is the lack of resources in the lower layers of a control system. These layers are those of the basic control loops and field devices with limited capabilities. CORBA has a great advantage over Java and DCOM as it has specialized specifications for vertical domains as that of the industry. For instance, the following are some of the available specifications in the industrial domain. • • • • Real-Time data distribution [OMG 03e]. Data acquisition from industrial systems [OMG 02c]. Historical data acquisition from industrial systems [OMG 03f]. Utility management systems data access facility [OMG 02d]. Even more, the current ongoing research on real-time component models focuses mainly into the CORBA world as Microsoft is not interested in real-time and the development on distributed real-time Java has been stopped for a long time now. 68 Chapter 4 CORBA Systems 69 4.1 CORBA Technology This chapter introduces the basics of the technology employed for CORBA Control Systems. The OMG has produced and continues to evolve a huge amount of specifications related to CORBA. Many of them are interesting to the distributed control field while others are more focused on business or enterprise applications. In the next sections, the most relevant aspects and topics of CORBA technology for distributed control applications are presented. There is information in this chapter about the following topics. • • • • • • • • The Object Management Group (OMG). As the organization leading the CORBA and other important specifications (e.g. UML) and standards (e.g. the IDL standard). The Object Management Architecture (OMA). As the reference architecture for CORBA Control Systems. CORBA Object Services (CORBAservices). As the basic building blocks of CORBA Control System distributed applications. The CORBA Component Model (CCM): As the next step in future CORBA applications. Real-Time CORBA. As the underlying framework for distributed applications in which control of resources can be exercised for end-to-end predictability. Minimum CORBA. To be employed in resource-limited devices of control systems. Data Distribution Service. For the dissemination of information in a distributed system with real-time requirements. Data Acquisition from Industrial Systems (DAIS) and Historical DAIS. Describes and interface for data, event and alarm access from industrial systems. The HDAIS specification defines an API for its use with time series data in industrial systems. 4.2 The OMG The Object Management Group (OMG) is a not-for-profit organization established in 1989 with the goal of developing vendor independent specifications to foster object technology by means of the creation of a software marketplace for object technology. It aims to reduce the complexity, lower the costs, and hasten the introduction of new software applications and systems. By using OMG's object technology any 71 organization can leverage previous efforts in building distributed object systems. The OMG is a standardization organization with an open, vendor-neutral, international, widely recognized and rapid standardization process based on demonstrated technology. Nowadays, software vendors, developers and users working in various different fields belong to the OMG membership as well as universities and governmental institutions. Further, it maintains a strong liaison with other organizations like ISO, ITU-T, W3C, TINA-C, and Meta Data Coalition. OMG's object technology is the object technology of reference: CORBA, IDL, UML, MOF, XMI, MDA, etc. Their standards allow interoperability and portability of distributed object oriented applications of different vendors. They do not produce software or implementation guidelines; only specifications which are put together using ideas of OMG members who respond to Requests For Information (RFI) and Requests For Proposals (RFP). The strength of this approach comes from the fact that most of the major software companies interested in distributed object oriented development are among OMG members. 4.3 Brokering Architectures Brokering architectures are those based on an Object Request Broker (ORB). In words of the OMG, an ORB is the middleware that establishes client-server relationships between objects. By means of the ORB, clients can transparently invoke methods on objects, which can be on the same machine or across a network. ORBs receive client calls and it is under their responsibilities to find the object that implements the requests, pass it the request parameters to invoke the method and to return the reply if any. For the clients there are several levels of transparency. The client is not aware where the object providing the service is located or the programming language it is implemented in. Neither it is aware of the operating system or network protocol used to access the object. The client is only aware of the object’s interface, which is all needed to access the object. The ORB is responsible for resolving all the details regarding how the object is reached, providing in this way interoperability in heterogeneous distributed environments. For interoperability to be possible between different hardware devices, programming languages, operating systems and networks an interoperability protocol independent of all the former conditions is needed. In an ORB, the interoperability 72 protocol is simply defined by the application interfaces which are expressed in a language-independent interface specification language, the IDL. IDL is good because it concentrates on the important part, what the offered services are. In the case of OMG’s IDL, the ISO standard IDL, it is as CORBA object-oriented. But this does not mean integration is only possible within an object-oriented framework. One of the first objectives of an ORB is integration. Within an ORB, integration is possible even with old legacy systems which are not object-oriented (let’s say a COBOL system) because the only thing IDL specifies is the interface. It does not specify how the system is implemented. Legacy systems can be integrated into and ORB-based or in a brokering architecture by the use of interface wrappers. Further, the OMG defines CORBA mappings to non-object-oriented languages. The most worldwide spread brokering architecture is OMG’s OMA, the Object Management Architecture from the OMG which is treated in the next section. 4.4 Object Management Architecture The Object Management Architecture (OMA) [OMG 96a] belongs to the main contributions of the OMG to the OO world. This is a specification for the construction of open distributed object systems based on brokering and on a collection of predefined services. It is a high-level vision of a complete distributed environment and consists of four components that can be roughly divided into two parts: system oriented components (Object Request Brokers and Object Services), and application oriented components (Application Objects and Common Facilities). The items below describe the main building-blocks of the OMA: • • • Object Request Broker (ORB): the central building block of the OMA that is the run-time integration vehicle for forwarding requests and responses between CORBA objects. Common Object Services (now called CORBAservices): provide fundamental services that are near to the system level (e.g. Naming Service and Notification Service). Horizontal CORBAfacilities: services that do not fit into a particular vertical market but are at a too high level to be called CORBAservices (e.g. the Printing Facility, the Secure Time Facility, the Internationalization Facility, and the Mobile Agent Facility). 73 • • Vertical CORBAfacilities (or Domain CORBAfacilities): provide standardized services for a particular field of application (e.g. Healthcare and Transportation, Utilities or Telecommunications) Application Objects: provide a particular service for a specific customer. The Object Request Broker (Figure 26) is the one of these parts which constitutes the foundation of the OMA and manages all communication between its components. It allows objects to interact in a heterogeneous, distributed environment, independently of the platforms on which these objects reside and of the techniques and languages used to implement them. In performing its task it relies on Object Services which are responsible for general object management such as object creation, access control, etc. and to provide an environment in which single objects perform their tasks. Common Facilities and Application Objects are the components closest to the end user, and in their functions they invoke services of the system components. These types of facilities can be configured to the purpose of a specific application. Application Objects Vertical CORBA Facilities Horizontal CORBA Facilities Object Request Broker CORBA Services Figure 26: The CORBA OMA When the OMG designed the OMA, it had to establish a balance, from the software architecture point of view, among its quality attributes [Bass 98]. The basic attributes it tried to address are: • • Buildability. The OMG tried to specify an architecture which could be feasibly implemented with available technology. Buildability is managed by the OMG standardization process that requires commercial implementations of the specification be built within one year of their approval. Balanced specificity. The architecture should be detailed enough to provide a standard that could be implemented by developers but general enough to 74 • • • • avoid vendor lock-in and to allow market competence by optimizations and specific features. This attribute is represented by the OMA ability to represent object interfaces in IDL. Implementation transparency. The architecture should provide transparency from implementation details so application clients can be independent of object implementation details. Interoperability. This is addressed by the interoperation of objects developed on OMA implementations from different vendors and by the implementation of bridges to other technologies. Evolvability. The architecture should be able to accommodate new distributed system technologies and needs from the market (this is achieved by the object services and facilities of the OMA). OMG’s Task Forces and special-interest groups are in charge of the evolvability process of the OMA. Extensibility. The architecture provides a nucleus of features which are useful and needed by most distributed application developers but is open enough to accommodate specific niche features (room for extensibility is provided by vertical domain facilities). The OMA provided a balance by making a trade-off of all the above attributes which are in tension by each other; a general but detailed architecture, general but niche optimal, stable versus flexible and so on. The trade-off performed on the quality attributes of the OMG brokering middleware is reflected on how the OMA is partitioned into architectural elements. CORBAservices are useful to most developers while specific domain vertical facilities are good for specific niches. All these services are independent of those of the ORB which are needed by services and facilities (which also are CORBA objects) and from application objects which are application specific. 4.5 Services and Facilities Of special interest are the CORBAservices to all distributed applications. Services provide pre-built functionality for the construction of applications from CORBA building blocks. They are classified as CORBA services as they are useful in a wide range of different domains (horizontal services) and should be used whenever possible to enhance application reliability. Next a description of some of the most interesting CORBA object services is presented. • Collection Service: The Collection Service [OMG 02e] provides a uniform way to manage the most general types of collections. Collections are groupings of objects and the operations needed to manipulate them as a group. 75 • • • • • • The Collection Service supports three types of interfaces in order to manage collections; collection and collection factory interfaces, iterator interfaces and, function interfaces. Concurrency Service: The purpose of the Concurrency Service [OMG 00] is to mediate concurrent access to an object such that the consistency of the object is not compromised when accessed by concurrently executing computations. The Concurrency Service uses a Lock Model in order to guarantee serialized access to objects. Enhanced View of Time: This specification [OMG 02a] leaves the earlier Time Service Specification [OMG 02f] unchanged and presents a service in which clocks with properties different of those of the Time Service can be used. The main advantage of the specification is its ability to represent clocks with differing characteristics (resolution, precision, stability or offset, skew and drift if more than one clock is used in the system). Event Service: There are situations in which it is necessary to decouple CORBA clients and server objects. The Event Service [OMG 04] provides the roles of event supplier (data producer) and event consumer (data processor) in order to provide decoupling between objects. Data is communicated between suppliers and consumers by issuing CORBA requests. The service supports two event communication models. The push model and the pull model. In the push model a supplier of events is able to initiate the communication of data to the consumers. In the pull model the consumer initiates the communication of the event. An event channel is a CORBA object that allows asynchronous communication of events between multiple suppliers and consumers by using standard CORBA requests. Externalization Service: The Externalization Service [OMG 00a] provides conventions and protocols for externalizing and internalizing objects. To externalize an object is to record the object’s state in a stream of data. Clients can request that externalized data be stored in a file using a standardized format that is defined as part of the Externalization Service specification. Internalizing is creating a new object from the data stream of an externalized object. Licensing Service: The Licensing Service [OMG 00b] provides a mechanism for producers to control the use of their intellectual property in a manner determined by their business and customer needs. It is important to notice that no products with temporary or expiring licenses can be selected for mission critical applications. Life Cycle Service: The Life Cycle Service [OMG 02g] provides a set of interfaces for creating, deleting, copying and moving objects. The service 76 • • • • • • allows clients to perform life cycle operations on objects residing in different locations. Lightweight Services: These are specifications of services for resource constrained systems. There are Naming, Event and Time services lightweight specifications [OMG 04a] and a Lightweight Log Service [OMG 03g]. The log service is designed as a central facility inside an embedded or real-time system capable to accept and manage logging records. Records are stored in a memory area and are owned and managed by the Lightweight Logging Service. Naming Service: The Naming Service [OMG 02h] allows for humanreadable naming of objects. Names can be associated to object IORs (binding) and object references can be obtained by the use of the object name (resolved). The Naming Service can be thought of as a “yellow pages” service for CORBA objects. It is based upon the name-to-object association which is called name binding. Name bindings are contained inside Naming Contexts which are CORBA objects that manage a set of name bindings. Each name binding is unique inside a naming context. Notification Service: The Notification Service [OMG 04b] extends the Event Service providing new capabilities. Among these new capabilities are the functionality to transmit events in the form of well-known structures, the ability for clients to receive specific events, the ability for suppliers to inquire about the existence of consumers so they can provide events on demand (this puts some leverage to the bottleneck problem that can occur when too many suppliers and consumers use an Event Service). Persistent State Service: The Persistent State Service [OMG 02i] provides interfaces for the retention of the persistent state of objects. This service goes further than the object reference persistent storage that CORBA provides (object_to_string) and should be capable of restoring the dynamic state of an object from its persistent state. The Persistent Object Service uses the Externalization Service as a POS (Persistent Object State) protocol. Property Service: The Property Service [OMG 00c] provides support to dynamically associate objects to properties which are typed named values (tuples <name, value>). The object properties can be used for multiple purposes (e.g. object classification regarding some property). Properties are not part of the object type; instead it is information of an object maintained by the service. Properties are the dynamic equivalent of CORBA attributes. Query Service: The Query Service [OMG 00d] provides query operations on collections of objects. Queries are predicate-based and may return collections of objects. The collections returned by the Query Service may be selected from a source of collections based on whether their member objects satisfy a 77 • • • • • • given predicate or may be produced by query evaluators based on the evaluation of a given predicate. Relationship Service: Objects are frequently used to model entities of the real world. Thus, objects do no exist alone but related to others. The Relationship Service [OMG 00e] allows entities (CORBA objects in a distributed system) and relationships to be explicitly represented. Security Service: The Security Service [OMG 02j] enables a distributed CORBA system to have control over identification, authentication, authorization and access control. It also provides interfaces for security auditing, administration, security of communication between objects and nonrepudiation functionality. Telecom Log Service: The Telecom Log Service [OMG 03h] extends the present Log Control Function specification (ITU-T X.735/ISO 10164-6) of the International Telecommunication Union which allows logging any type of event and querying of the log records by the use of a constraint language. Time Service: The Time Service [OMG 02f] enables its clients to obtain current time along with an estimate of the error associated with it. Unfortunately, the time representation (UTC from the X/Open DCE Time Service) of the Time Service Specification is not well suited for some fields of application. Other specifications as the Smarts Transducers Specification make better choices as their representation of time is synchronized with that of GPS and have a granularity of up to 60 ns. Trading Object Service: The Trading Service [OMG 00f] enables to offer and to discover instances of services of particular types. A trader is an object which supports the Trading service and can be considered as a whiteboard where other objects can either advertise their capabilities or search for objects which advertise certain features. Transaction Service: The Transaction Service [OMG 03i] defines interfaces that allow multiple, distributed objects to collaborate to provide atomicity. The service interfaces allow the objects either to commit all changes together or to rollback all the changes together. 4.6 Real-Time CORBA There are a great number of distributed systems which are real-time systems. These are systems which have requirements regarding the end-to-end predictability and the Quality-Of-Service (QoS) of their activities. Examples of this type of systems can be found in the process control, command and control, avionics or telecommunication fields among others. If advantage is to be taken from the benefits of using an ORB for 78 the development of real-time distributed applications, the requirements for that type of systems must be taken into account at the middleware level. RT CORBA [Schmidt 00] specifies additional mechanisms to those of CORBA that can be employed to increase the control of resources and end-to-end predictability of distributed object applications. Figure 27 provides an overview of the extensions compared to traditional CORBA. The following sections provide a description of the key entities and features of a real-time CORBA broker as addressed in the Real-Time CORBA specifications (see [OMG 05] and [OMG 03]). There are currently two specifications for real-time; one addresses fixed priority scheduling [OMG 05] and the other addresses dynamic scheduling [OMG 03]. Although an overview is given for both of them, in practice, most real-time ORBs conform to the fixed priority scheduling specification. Scheduling Service Client Server RTCORBA::Current POA RTPOA CORBA::Current RTCORBA::Priority RTCORBA::Threadpool ORB + RTORB IIOP RTIOP RTCORBA::PriorityMapping Figure 27: Real-Time CORBA extensions provide strong control over application resources on the client and server sides 4.6.1 Fixed-priority Real-Time CORBA The first real-time CORBA specification did not address the problem of dynamic scheduling. It only supports fixed-priority scheduling. The objective of the specification is the control of resources to enhance end-to-end predictability in a distributed real-time system. Regarding end-to-end predictability the specification is defined to mean: • Respecting thread priorities between clients and servers for the resolution of resource contention during the processing of CORBA invocations. • Bounding of thread priority inversions during end-to-end processing. 79 • Bounding the latencies of operations invocations. The specification relies in an extension of key CORBA entities (ORB, POA, Current, threading model, etc.) and on the services provided by them. The Real-Time CORBA specification allows managing three types of resources in a distributed system. • • • 4.6.2 Processor resources. Communications resources. Memory resources. RT-ORB The real-time ORB exploits an extension of the CORBA::ORB interface that provides operations to create and destroy constituents of a real-time ORB. A real-time ORB incorporates functionality that allows the application running on top of it to specify the information related to QoS or resource requirements needed for the correct operation of the system. For example, it is possible to configure the range of thread priorities the ORB is allowed to use during execution. 4.6.3 Real-time POA In much the same way a real-time ORB must be able to process the real-time requirements of the objects on top of it, the Object Adapters running on its server side must also understand how to process those requirements. The Portable Object Adapter for Real-time CORBA is defined in the module RTPortableServer. The Real-Time POA is a subtype of the CORBA PortableServer::POA. The POA for real-time has two different characteristics from that of CORBA: • • It should understand the real-time policies specified in the real-time extension. It provides an additional set of operations to support object level priority settings. The Real-Time POA groups a set of operations designed to override the server declared priority on a per-object reference basis. Examples of these operations are create_reference_with_priority or activate_object_with_priority that allow setting an execution priority for CORBA objects or interoperable references. A common behavior of real-time POAs is to encapsulate the values regarding to server-side real-time policies into interoperable object references (IORs) so it is possible for the clients accessing those objects to honor the real-time behavior expected by them. 80 4.6.4 CORBA Priorities Different RTOSs show different native thread priority schemes. As a result of RTOS heterogeneity, different native priorities exist which represent a problem when trying to establish a common behavior in a real-time heterogeneous distributed system. RealTime CORBA solves this hindrance by defining a CORBA Priority type which is valid for the entire system. As in the RTOS native priority schemes, CORBA Priority is a set of integer values that maps the native priority scheme of a specific RTOS to a uniform scheme which is accepted system-wide. The Priority for real-time is defined in the RTCORBA module. Real-Time CORBA also defines a PriorityMapping object which is defined as an IDL native type. Native types are a special type definition in CORBA that result in specific implementations in each programming language. A PriorityMapping is a programming language object (an instance of the RTCORBA::PriotityMapping class in C++) instead of a CORBA object. Consequently, mappings to programming languages (C, C++, Ada and Java) are also provided in the submission for Real-Time CORBA. Figure 28 shows a mapping of priorities between Real-time CORBA and two particular operating systems. Notice that not all the priorities have to be mapped from CORBA to the OS. The priority mapping interface allows to perform the mapping of CORBA priorities to the native values of the operating system and from the operating system to CORBA. RTCORBA::Priority 31 0 0 31 25 4 255 OS #1 native priority model 0 OS #2 native priority model Figure 28: CORBA Priorities and mapping to OS CORBA priorities are used to allow the clients and CORBA objects specify the priorities at which requests must be executed. This allows enhancing end-to-end predictability in a fixed-priority scheduling mechanism. 81 4.6.5 Real-time Current Interface Real-Time CORBA derives the real-time Current interface from the CORBA::Current interface. The specific objective of Current for real-time is to obtain the CORBA Priority of the current thread. The RTCORBA::Current object contains a priority attribute which can be set and consulted. As a result of setting the priority attribute, the ORB sets the native priority of the thread to the value returned by PriorityMapping::to_native. With this feature, the process under execution can dynamically act upon the thread priority it has been assigned by the ORB. 4.6.6 Real-time CORBA Priority Models and Transforms In the context of a real-time distributed system there must be resources that allow the infrastructure to enforce thread eligibility of execution on remote objects. For this purpose, Real-Time CORBA supports two types of priority models to coordinate priorities across systems (see Table 3). The Priority Model along with the use of CORBA Priorities is designed to minimize end-to-end priority inversion as well as to bound latency and jitter for applications with real-time requirements. Priority Model Client Propagated Priority Model Server Declared Priority Model Description Priority is carried with the CORBA invocation and is used to ensure that all threads subsequently executing on the invocation run at the appropriate priority. In this model, objects publish their CORBA priorities in their object references. This lets clients know the priority of invocation execution in the servant’s code. The Real-Time CORBA Priority of an invocation needs not be passed from client to server in an invocation. Table 3: Real-Time CORBA Priority Models Additionally, it is possible to override a Server Declared Priority on a per-object reference basis. The Real-Time CORBA POA interface provides four operations for doing this. The server priority can be changed at the time of object reference creation or at the time of object activation. Priority Model Policies must be applied to POA objects at the time of creation. To let clients know the policy models supported by CORBA objects, the policies are propagated from servers to clients within IORs. The mechanism of propagation is 82 defined in the Quality of Service framework of the CORBA Messaging specification. Figure 29 and Figure 30 show how the Client and Server Priority Models work. Transforms are user-defined priority transforms that modify the CORBA Priority associated with an invocation. The transforms take place when the invocation is processed by a server. Priority transforms map Real-Time CORBA priority values to other application-defined CORBA priority values. These can be used to implement specific priority protocols. As PriorityTransform is an IDL native type, language mappings are provided for different programming languages (C, C++, Ada and Java) in the Real-Time CORBA specification. There are two types of priority transforms: inbound and outbound. Inbound transforms take place when an invocation is received by a server but before it is processed by a servant. Outbound transforms are applied to ongoing requests from a servant to other CORBA objects. Invocation executed at priority 100 in the server Client running at priority 100 Servant Client Stub Skeleton Client’s priority propagated along the path to server in a service context ORB POA ORB Figure 29: Client propagated priority model Invocation executed at priority 3347in the server Client running at priority 100 Servant Client Stub Skeleton Client’s priority IS NOT propagated along the path to server in a service context ORB POA ORB Figure 30: Server declared priority model 83 Priority transforms are useful because there are situations in which the server or client declared priority models are not enough. The advantage of priority transforms is that they allow the application of the priority of execution taking external factors into account. For instance, the occurrence of a critical external event or the workload of the server computer could be used to decide on the priority of the request. 4.6.7 Synchronization Synchronization is the satisfaction of restrictions that exist in the interleaving of actions of different processes or threads [Burns 97]. One of the most difficult parts of building a concurrent application is dealing with process or thread interaction. Threads and processes are not independent of each other and system’s behavior depends on their synchronization and communication. For our purposes, communication can be understood as the passing of information between different processes or threads. This can be achieved by means of shared variables or message passing. Real-Time CORBA provides a mutex interface to allow threads execute in regions protected by mutex objects. Objects of mutex type provide the degree of synchronization needed to protect a critical section (this is known as mutual exclusion and the MutEx word reflects that name). Real-Time CORBA provides two operations in the RTORB interface to create and destroy mutexes, so the same mechanisms than the broker uses for synchronization can be used by the application on top of the ORB. This helps to reduce priority inversion as all the mutexes will use the same protocol (e.g. a priority ceiling protocol). 4.6.8 Handling Concurrency In the real world things happen in parallel. The term concurrency refers to an expression of the parallelism present in the world. In the computer, parallelism is achieved by concurrent programming. The term “concurrent” does not mean parallel but “potentially parallel”. This is because concurrent programs or applications are formed by a set of sequential processes that are (logically) executed in parallel [Dijkstra 68]. Parallelism depends on how the collection of processes is executed. • • • The execution is multiplexed in one processor. The execution is multiplexed in a multi-processor system where memory is shared (parallel computing). Execution is multiplexed in several processors and no memory is shared (distributed system). 84 Concurrency can be expressed either at the level of processes (programs) or inside the program (threads). In the latter case memory is shared by the different threads of the program. Multi-threading is usually employed in addition to providing virtual parallelism to allow the application increase its control over how different types of services are executed (e.g. high-priority versus low-priority services) and to support thread preemption to avoid priority inversion during lengthy request invocations. OS APIs to create threads are not always standard (it is difficult to find full implementations of the POSIX threads interface in all platforms) and a model of thread interface (a wrapper class) is useful when developing for several platforms. This is what Real-Time CORBA defines, a thread and a grouping of threads model that allows heterogeneous distributed applications to be multithreaded in a simple, uniform manner. Real-Time CORBA introduces the threadpool concept (the grouping of threads) which in a literal and straightforward definition is understood as a pool of threads. There are two possible styles for the threadpool; with and without lanes (Figure 31). A lane is a subset of the threadpool in which all the threads have the same RTCORBA::Priority value. Different lanes in the threadpool may have different priority values. Lane A Lane B Threadpool Threadpool Figure 31: Threadpool models, with and without lanes The operations for the creation of threadpools allow to configure the stack size and request buffering for the threadpool. The stack size is important because operation arguments are stored in the thread stack and enough memory to avoid access violations must be allocated for this purpose. As with other policy types, a threadpool can be associated with several POAs at creation time (Figure 32) by using a ThreadpoolId (an identifier assigned by the ORB to every threadpool). Other interesting features can be set in the threadpool creation operations. • Static threads: If lanes are not being used, it is the number of threads to be assigned to the threadpool. In a threadpool-with-lanes style it designates the number of threads in each lane. 85 • • Dynamic threads: It is the number of threads that can be created dynamically and that are allocated either to the threadpool or to the lane. Priority: Without lanes it is the CORBA priority assigned to the static threads in the threadpool. In this case, dynamic threads can be created at the priority required to handle the invocation they were created for. If lanes are being used, the CORBA priority is assigned to all the threads (statically and dynamically allocated) in the lane. In the styles with lanes, borrowing of threads from lower priority lanes can also be specified. Once the borrowed thread has finished the request it is returned to the original lane. POA A POA B POA A POA B Threadpool A Threadpool B threadpool ORB ORB Figure 32: Assignment of threadpools to real-time POAs 4.6.9 Handling of Connections Predictability is a main issue in a real-time system. In a conventional ORB, binding does not occur until the first invocation on an object is made. In CORBA this is known as implicit binding. In an implicit binding approach, communication resources along the request path are established on demand, they are requested when they are needed. The binding is a source of overhead that puts a penalty on the first invocation made by a client. Real-Time CORBA anticipates this problem by making allowance for explicit binding. Client applications can control the time when the binding on an unbounded object reference is made by means of the validate_connection operation. As with all models, implicit and explicit binding have their advantages and limitations. Implicit binding helps the developer to conserve system resources and to share connections among different clients or execution threads. However, this model is not sufficient for real-time and control applications and a better way of managing resources is needed. This is what the explicit binding model provides. In the explicit 86 binding model, connection to objects can either be deferred until the execution of the request (as in the implicit binding) or resources can be reserved in advance to requests (pre-establishing connections to server objects). This reduces latency and jitter for the first invocation. Further, it is possible to use private connections or to assign a range of priorities to a connection reducing also the possibility of priority inversion that occurs in a multiplexed connection approach. To support the explicit binding model, Real-Time CORBA provides the Priority Banded Connections feature. A Priority Banded Connection is a connection with an assigned set of PriorityBands. Priority bands assigned to a connection cannot be reconfigured during the lifetime of the connection and no priority may be covered more than once. As Figure 33 shows, it is also foreseen that non-contiguous ranges can be formed and that it is not necessary to cover all CORBA priorities. Banded Connections: Invocation priority determines connection to use client client client server client 0 - 1300 ORB ORB 9786 - 21340 Figure 33: Priority bands and explicit binding The idea of banded connections is to allow clients communicate with servers using multiple connections reserved for invocations made at different CORBA priorities. If the priority of invocations is not respected by the transport it will become a source of priority inversion. The connection is chosen depending on the target object priority model. In the case of the Client Propagated priority model, the band is chosen using the priority specified by the client. If the Server Declared priority model is being used, its priority is published in the IOR and its value is used to select the band. 87 Banded connections are configured by clients and the policies are applied to the client side only. By default, the ORB provides a multiplexed connection for client / server connections (Figure 34). However, it is possible to request a private transport connection (Figure 34) by means of the PrivateConnectionPolicy object. The advantage of private connections is that it ensures that no clients will interfere with other requests in the case the reply from the server is needed. Reserving a private connection allows to maintain the communications channel free of traffic while the client is waiting for the reply to the request. client client Multiplexed: client Resource contention by shared connection client ORB ORB client client Private connection: Exclusive use of a connection by a client client client ORB server server ORB Figure 34: Multiplexed and private connections in Real-Time CORBA 4.6.10 Invocation Timeout Bounding the time in which a reply from a server must be obtained is a useful tool for predictable system development. Predictability can be improved if system developers know the upper bound of the time an invocation will be blocked waiting for the server to answer. This functionality is achieved setting timeouts. Real-Time CORBA uses the Messaging::RelativeRoundTripTimeoutPolicy to set the timeout for the reception of a reply to an invocation. In the case of Real-Time CORBA the timeout policy is a client-side only policy. It is applied to clients who will cancel the request if the reply has not arrived before the 88 timeout expires. If the reply arrives at the client after the timeout has expired it is simply discarded by the ORB. 4.6.11 Protocol Configuration Real-Time CORBA applications may find that best-effort QoS requirements do not meet their needs. In the general CORBA case, applications need not worry about the location transparency in what refers to the protocol used for communications. Anyhow, real-time systems like control systems need a tighter command and selection of the protocols used for communication in order to improve end-to-end predictability. Real-Time CORBA specifies a programming interface to select and configure the underlying communication protocol (Figure 35). In CORBA, IOP instances (the object that represents the interoperability protocol) contain an ORB protocol and a mapping to an underlying transport protocol. Notice that two protocols are used, the ORB message protocol (GIOP in the CORBA standard) and a transport protocol (TCP/IP as prescribed by CORBA). The combination of both protocols results in the Internet Inter-ORB Protocol of CORBA or IIOP, the standard CORBA interoperability protocol. To provide control on the protocols used by the application, Real-Time CORBA defines an interface that allows applications to control protocol properties. It is possible to configure protocol properties either at the client-side or at the server-side of an RT ORB. Object Reference Servant Client Skeleton Stub ORB A ORB B RTOS A RTOS B Invocation ATM TCP/IP Figure 35: CORBA TCP/IP Protocol configuration 89 in ATM Other Real-time Protocols are specified by means of a protocol list which can be applied to POAs. The list indicates the protocols supported by a certain POA and the order of the protocols in the protocol list indicates the order of preference for the usage of protocols. The policy is client-exposed, meaning that it is encapsulated in IORs to be consulted by clients making invocations. On the client-side, Real-Time CORBA defines a similar interface for the client protocol policy. The difference is that the policy is applied at the time of binding to an object reference. In the server side it is applied at the time of POA creation and the policy is propagated from server to client in the IORs. 4.6.12 Real-Time Scheduling Service The real-time scheduling service is a tool that enforces a certain scheduling policy to be used across the whole real-time CORBA system. It is useful because specifying the appropriate configuration parameters in all parts of the real-time system may be a complex task. The scheduling service provides a form of central repository from where a uniform scheduling policy for the whole system can be obtained by CORBA objects. 4.7 Dynamic-Scheduling Real-Time CORBA In order to generalize the Real-Time CORBA specification and meet the requirements of a greater field of applications in real-time computing, another specification (dynamic scheduling, see [OMG 01b] and [OMG 05]) has been created. The RealTime CORBA 2.0 specification tries to address static and dynamic distributed systems. Dynamic distributed systems are those in which the processing workload of the system is not well known before-hand or no bounds can be put to it so it is not possible to perform an off-line analysis of the system. The specification generalizes the concepts of distributed system scheduling and that of a distributable thread in order to allow the application or the ORB have control on the following features. • Any scheduling discipline may be employed. • The scheduling parameter elements associated with the chosen discipline may be changed at any time during execution. • The schedulable entity is a distributable thread that may span node boundaries, carrying its scheduling context among scheduler instances on those nodes. In order to provide more control over the temporal behavior of the application, it interacts with the scheduler than can use one or more scheduling disciplines such as 90 Fixed Priority Scheduling, Earliest Deadline First, Least Laxity First, Maximize Accrued Utility, etc. [Liu 00], [Burns 97]. The specification provides IDL interfaces for all the above scheduling disciplines but it is possible to establish any other scheduling discipline. The goal of the scheduler is to determine how best to meet the schedule given a predicted use of system resources by the application in a certain instant of time. The specification provides a framework in the form of IDL interfaces that allows the development of portable schedulers. 4.7.1 Distributable Thread In dynamic Real-Time CORBA, the notion of activity from Real-Time CORBA 1.0 has been replaced by that of distributable thread. For dynamic systems and in order to achieve end-to-end timeliness a trans-node application behavior must be enforced. This can be done by using the time and resources related parameters in a consistent system-wide manner for allocation of resources. A trans-node application behavior abstraction is defined (the distributable thread) for this purpose. A distributable thread is a programming model abstraction. It is a thread that can execute operations in objects without regard for the physical node boundaries (Figure 36). The distributable thread is the schedulable entity in this specification. Control flow Object A Object B Object C Figure 36: Control flow in a distributed processing system Each distributable thread may have several execution parameters (e.g. priority, deadlines, utility functions, etc.) to specify the end-to-end timeliness in order to complete the set of sequential operations in objects that may reside in different physical nodes. The distributable thread transports scheduling information across the distributed system. 91 Distributable Thread Forking Forking is the part of a concurrency model that deals with the creation of a new execution context. The specification allows for explicit forking of a distributable thread by the use of the spawn operation. An example of forking is that of one-way invocations as the distributable thread making the invocation is not blocked waiting for the servant to process the invocation. Scheduling Segments A distributable thread consists of one or more scheduling segments. A scheduling segment represents a sequence of control flow related to a set of scheduling parameters. A scheduling segment has only one starting point and one ending point which may span processor boundaries. Distributable Thread Traversing CORBA Objects Segment Scopes BSS W Seg. W BSS X Seg. X ESS W BSS Z ESS X Seg. Z ESS Z Object A Object B BSS- Begin Scheduling Segment USS- Update Scheduling Segment ESS- End Scheduling Segment Object C Portable interceptor Application call Figure 37: Distributable thread with nested scheduling segments Figure 37 shows an illustration of a distributable thread with nested scheduling segments. Where the segment X begins, the scheduling context of segment W is put aside and segment’s X scheduling parameter elements are used for the distributable thread. When segment X ends, the scheduling parameters for segment W are used until it ends. 92 4.7.2 Real-time CORBA Scheduler The Scheduler is an extension to Real-Time CORBA that manages the scheduling requirements and parameters of the applications that run on top of the broker. The scheduler decides the order of execution of the applications on the distributed nodes of the CORBA system. To decide on the execution eligibility in the CORBA systems the scheduler is based on the following characteristics. • The scheduler responds to application requests (to define scheduler elements) and in response to application actions (e.g. such as invocations by using the Portable Interceptor interfaces). • The scheduler uses the information provided by the application to decide on the eligibility of threads. • The scheduler architecture is based upon the concept that the distributed system can be considered as a set of distributable threads. • It is supposed that the schedulability of the system can be addressed by managing the allocation of resources to distributable threads. • The distributable threads and the scheduler interact at specific scheduling points such as in transitions to new processors where scheduling information must be re-interpreted. • The scheduler is a pluggable scheduler. If an ORB has a scheduler installed, all applications that run on that ORB use that scheduler. Scheduling Points The scheduling points are the points in time and/or in code where the scheduler is run. This may result in a change of the current schedule. The defined scheduling points are shown below. • Creation of a distributable thread. • Termination or completion of a distributable thread. • Beginning of a scheduling segment. • Update of a scheduling segment. • End of a scheduling segment. • A CORBA operation invocation, specifically the request and reply interception points provided in the Portable Interceptor specification. • Creation of a resource manager. • Blocking on a request for a resource. • Unblocking as result of the release of a resource. 93 Schedule-aware Resource The specification allows the creation of schedule-aware resources via a Resource Manager. The resources can have scheduling information associated with them. 4.7.3 Advice about Real-Time CORBA Originally, CORBA was not planned to be deployed in hard real-time systems. Therefore several sources of non-determinism can be identified that could cause the system to miss its deadline: marshalling of parameters at the client side, protocol queues of the client, delays on the transport media, protocol queues on the server, dispatching of threads and requests, and un-marshalling of parameters at the server side. Nevertheless, there is no clear way of reducing complexity by subdividing a system into subsystems and retaining the temporal behavior. Thus, the implementer is left with the complexity of the whole system. Introducing composability while preserving end-to-end predictability is still an open issue. Especially analysis of a RT-System according to the real-time CORBA specifications is complicated by features like borrowing threads among threadpools. 4.8 Minimum CORBA Minimum CORBA [OMG 02k] defines a cut-down profile of CORBA, not of Realtime CORBA, for those applications with limited resources. Regarding conformance to the full CORBA specification, Minimum CORBA follows these guidelines: The Minimum CORBA specification supports all OMG IDL to allow maximum compatibility between minimum and full CORBA applications. This point is important as most Real-Time CORBA brokers are built around a Minimum CORBA broker. Minimum CORBA applications must be fully interoperable with CORBA applications as Minimum CORBA applications may be part of a greater system running CORBA applications. Dynamic aspects of CORBA are omitted in Minimum CORBA as for the target applications of Minimum CORBA many decisions are made at design time. 94 4.9 Fault-Tolerant CORBA There are various kinds of applications with a need for fault tolerance, beginning from large critical systems (such as air traffic control systems) to smaller critical systems (such as medical systems) and embedded applications (such as manufacturing control applications). A standard that attempts to meet all of the requirements of this wide spectrum of applications might satisfy many needs only poorly, or might be too complex to implement. The Fault-tolerant CORBA specification [OMG 02b] therefore represents a number of compromises in order to support most of these systems. Fault tolerance depends on entity redundancy, fault detection, and recovery. The entity redundancy by which this specification provides fault tolerance is the replication of objects. This strategy allows greater flexibility in configuration management of the number of replicas, and of their assignment to different hosts, compared to server replication. Replicated objects can invoke the methods of other replicated objects without regard to the physical location of those objects. The standard supports a range of fault tolerance strategies, including request retry, redirection to an alternative server, passive (primary/backup) replication, and active replication which provides more rapid recovery from faults. Applications that require the Fault Tolerance Infrastructure in order to control the creation of the application object replicas are supported as well as applications that control directly the creation of their own object replicas and applications that require the Fault Tolerance Infrastructure to maintain Strong Replica Consistency, both under normal conditions and under fault conditions. Support for fault detection, notification, and analysis for the object replicas is supported. Thus, allowing the development of applications that require the Fault Tolerance Infrastructure in order to provide automatic checkpointing, logging and recovery from faults as well as applications that handle their own fault recovery. The standard aims for minimal modifications to the existing application programs, and for transparency to replication and to faults and, it defines minimal modifications to existing ORBs that allow non-replicated clients to derive fault tolerance benefits when they invoke replicated server objects. 4.9.1 Grouping Abstractions In Fault-Tolerant CORBA there are two types of grouping abstractions; the object groups and the fault tolerance domains. 95 Object Groups In order to achieve CORBA-objects fault tolerance, several replicas of the object are created and grouped in an Object Group. The object group can be addressed as a whole by the use of an Interoperable Object Group Reference (IOGR) which is exported by the server to be used by CORBA clients (client-exposed in OMG terminology). The IOGR is created by the Replication Manager (see next sections). The clients invoke their requests on the object group and are processed by the object members of the group. Each individual replica keeps its own IOR but the abstraction of the object group provides two advantages from the client point of view. • • Replication Transparency: The client objects are not aware that the server objects are replicated. Failure Transparency: The client objects are not aware of fault in the server replicas or of fault recovery. Fault Tolerant Domains FT CORBA also deals with large applications that have a need for fault tolerance. Such applications manage thousands of objects and span several locations so it is inappropriate to consider them as a single entity. To address this problem, fault tolerant CORBA defines the concept of fault tolerance domains (Figure 38). A fault tolerance domain usually contains several hosts and many object groups being possible for a single host to support several fault tolerance domains. TUVienna Location Host 3 A1 Host 1 IIOP Message over TCP/IP ORB without support for Fault Tolerance B2 E2 F1 C1 Host 7 Host 2 Gate way B1 Host 5 C3 D1 C2 Host 6 E3 F2 Host 4 SCILabs Domain E1 Wide Area Domain UPM Domain Figure 38: Fault tolerance domains All the objects and groups in a fault tolerance domain are managed by a single Replication Manager. Fault tolerance domains do not isolate objects in one domain from those of others. Objects in a domain can invoke to and be invoked by objects in 96 other domains. Figure 38 shows an example of fault tolerance domains which are filled in blue in the figure. Hosts are shown in green and members of an object group are shown in dark blue. The members of object group B are denoted as B1, B2. The same notation is valid for groups A, D, C, E and F. The fault tolerance domains allow applications to be arbitrary in size, putting no limit to application scale. This is achieved by letting replications managers handle a smaller number of objects than that of the whole system. Fault tolerance properties can be assigned either to object groups or to fault tolerance domains. Number of replicas or replication style (passive, warm passive or active) and other properties can be applied to all the objects of a group, domain or to all the object groups of a specific type. 4.9.2 Architectural Overview A fault tolerant CORBA system needs the infrastructure shown in Figure 39. A Replication Manager, Fault Notifier and a Fault Detector object are implemented as CORBA objects. Logically, only a single instance of these objects exists in a fault tolerance domain but they are physically replicated for fault protection as all the other objects of the application. The Replication Manager of Figure 39 inherits the Property Manager, Object Group Manager and Generic Factory interfaces. create_ object() set_ properties() Replication Manager notifications Fault Notifier create_ object() Fault Detector is_alive() fault reports H1 H2 H3 Client Server C CORBA Server S1 ORB Logging Mechanism S2 Factory Fault Detector Factory Fault Detector CORBA ORB CORBA ORB Recovery Mechanism Logging Mechanism Recovery Mechanism Logging Mechanism Figure 39: architectural overview of a fault tolerant CORBA system 97 The PropertyManager interface allows users to specify fault tolerance properties of object groups. Replication management is controlled by the use of the GenericFactory and the ObjectGroupManager interfaces. The GenericFactory interface is able to create replicated objects on application demand. The GenericFactory is not used directly; it is the Replication Manager who will invoke the factories on the hosts where the replica is to be created. The ObjectGroupManager operations are designed to add, remove or control the location of members of an object group. The figure shows three hosts H1, H2 and H3. The client C on H1 is invoking a replicated server with two replicas S1 on host H2 and S2 on host H3. The Factory and Fault detector objects in each host are not replicated as occurs with the service objects on top of the figure. All the application objects inherit a PullMonitorable interface that a Fault Detector invokes. It is a kind of watchdog invoking an is_alive() operation. Faults detected by in-host Fault Detectors are notified to the Fault Notifier which passes the notifications to the Replication Manager. In the case that the passive or warm passive replication styles are used, only one member of an object group executes the requests and sends the replies. On a faulty condition, the Replication Manager can restart the primary member of the object group or can promote a backup member to primary member. In the case of active replication all the members of an object group execute invocations independently and in the same order so as to keep the same state. When a fault occurs in one member the application continues with the results of other member without waiting for fault detection and recovery. There is a message handling mechanism that detects and suppresses all replies except one which is sent to the client. 4.10 Data Distribution Many real-time applications have a requirement to model some of their communication patterns as a pure data-centric exchange, where applications publish data which is then available to the remote applications that are interested in it. The Real-Time Data Distribution Service [OMG 03e] specification targets at solving the following problems: • Predictable distribution of data with minimal overhead since it is not feasible to infinitely extend the needed resources. 98 • Scalability to hundreds or thousands of publishers and subscribers in a robust manner. Data-centric communications decouples senders from receivers; the less coupled the publishers and the subscribers are, the easier these extensions for scalability become. The Data-Centric Publish-Subscribe (DCPS) model has become popular in many realtime applications. Compared to distributed shared memory which is a classic model that provides data-centric exchanges, this model is easier to implement efficiently over a network and allows the required scalability and flexibility. It builds on the concept of a “global data space” that is accessible to all interested applications. Applications that want to contribute information to this data space declare their intent to become “Publishers” and applications that want to access portions of this data space declare their intent to become “Subscribers”. Each time a Publisher posts new data into this “global data space”, the middleware propagates the information to all interested Subscribers. Another common need is a Data Local Reconstruction Layer (DLRL) that automatically reconstructs the data locally from the updates and allows the application to access the data as if it were local. In that case, the middleware not only propagates the information to all interested subscribers but also updates a local copy of the information. 4.11 CORBA Component Model Components are a way to increase productivity, reuse existing code and reduce complexity. The improvement of all those factors shortens the time to market of any product. Hence, the interest in the development of a component model for CORBA. Developing applications or systems with components can be thought of as assembling the application from pieces, with little development involved. Instead of engineering components, the focus can be put on the problem to solve. As the components are reused over and over they refine their quality, producing by composition systems of better quality. In the distributed object model of CORBA, the problems of heterogeneity, portability and interoperability are resolved in an elegant manner. All those problems that CORBA resolves can be summarized in one word: integration. CORBA is about integration. However, some problems like those explained in Section 3.6.3 prevent in some degree the development of a component market for CORBA. The CORBA Component Model (CCM) [OMG 02l] builds on the features of the CORBA distributed object model to deploy true components. 99 The distributed component model of CORBA defines an architecture for component definition and interaction, either from the client-side or the server-side (business components to build application servers). One of the critics of the CORBA object model was the lack of packaging and deployment facilities. In the CCM, a packaging and deployment model is defined to deploy binary multi-lingual executables. One of the complexities of CORBA was dealing with non-functional properties of objects (e.g. lifecycle of objects, persistence, etc.), in the CCM, a framework is defined in which non-functional attributes of components can be specified. CCM solves these problems without forgetting about being vendor-independent or multi-language or multi-ORB interoperable. The CCM specification deals with several aspects of CORBA components. • • • • • • 4.11.1 The CCM presents an abstract component model where components are represented as extensions to objects by extensions in the IDL and in the object model. The CCM defines a Component Implementation Framework which defines a Component Implementation Definition Language (CIDL). The CCM defines a Component Container Programming Model. The programming model is useful for the component implementer and for the client programmer point of view. The Container model integrates with nonfunctional features of components such as events, transactions, persistence and security. The CCM defines packaging and deployment specifications for components. It defines an interoperability specification with Sun’s EJB 1.1. It defines a MOF-compliant component meta-model to express the extensions to IDL of the CORBA Component Model and another one for the Component Implementation Framework. What is a CORBA Component? The Component is a basic CORBA meta-type that extends the functionality of the Object meta-type. As such, they can be expressed in IDL and represented in the Interface Repository. Components are represented by an object reference and its definition is made in an extended version of IDL. Components present surface features that allow them to interact with other elements of the application or clients. The surface features are called ports and exist in various forms. 100 • • • • • Facets. They are distinct named interfaces provided by the component for client interaction (services provided by the component). Receptacles. Which are named connection points that describe the component’s ability to use a reference supplied by some external agent (services requested by the component). Event Sources. Which are named connection points that emit events of a specified type to one or more interested event consumers, or to an event channel. Event Sinks. Which are named connection points into which events of a specified type may be pushed. Attributes. Attributes are named values intended for component configuration. In the CCM there are two types of components, basic and extended. A basic component can only have attributes whereas an extended component can have any type of port (Figure 40). Component interface Receptacles OFFERED CORBA Component Event sinks Event sources REQUIRED Facets Attributes Figure 40: Extended CORBA Component and ports 4.11.2 Limitations of the CCM for Distributed Control The CCM provides a standard way to deploy distributed systems with a great degree of software reuse and without the need of very skilled staff. While this is true for business-like systems, the construction of distributed control systems in this fashion is still a field of research. Today’s Real-Time CORBA ORBs can be used in a variety of 101 real-time systems with good results. This is the result of several years of development and improvement of the ORB QoS features needed in this type of domains. The same holds for a component approach to system development for control purposes. The CCM builds on top of CORBA which can be enabled to handle the requirements of control or real-time systems. Nevertheless, the CCM needs improvements to able to communicate these requirements to the underlying ORB and this is not dealt with in the CCM specification. Among the problems to be solved in order to use the CCM in distributed control systems there are the following. • • • The CCM has no mechanisms to enforce real-time policies. The CCM needs to able to configure the QoS control mechanisms of Real-Time CORBA. The CCM introduces new classes and types atop of CORBA. This results in more code to execute, introducing overhead in the time and memory needed by the component to perform the service. This is a problem in systems with tight requirements or scarce resources. The CCM is bulky and has a big memory impact. The Lightweight CCM specification [OMG 04c] tries to mitigate this problem. One of the advantages of the CCM is that it is able to deal with non-functional properties of the components as the events or transactions. A possibility for real-time systems is to extend the CCM to deal with the non-functional properties of the component’s real-time requirements. 4.11.3 The COMPonent Approach Embedded (COMPARE) Project for Real-time and The COMPARE project (IST-004669) is funded by the IST Sixth Framework Programme. The project consortium is composed by the following partners: Thales Communications, Schneider Electric, PrismTech, Trialog, CEA-LIST and UPM. COMPARE recognizes the limitations already presented of the component model of CORBA to be used in real-time and embedded systems and is focused at the specification and implementation of a component framework that allows dealing with the complexity and performance issues of this type of systems. COMPARE starting point is that complexity is already addressed in other types of systems (mainly business systems) by the use of component frameworks (e.g. SUN’s EJB). COMPARE tries to apply this same idea to the area of real-time and embedded systems by tailoring and extending the Lightweight CORBA Component Model. This means studying the LwCCM to make the necessary modifications. Concretely, the following items are addressed (see Figure 41). 102 • • • • Interaction models for the adaptation of the LwCCM to the RT/E area. Current supported interaction models are synchronous call of methods and asynchronous sending of events (messages). For RT/E systems, other models such as streaming ports are worth to study. In any case, Quality of Service should be supported for any of these interaction kinds. Limitations of LwCCM Containers. The currently supported services are not suitable for RT/E and work is needed in areas regarding real-time properties (e.g. scheduling support), proper activation of components (e.g. periodic activation), etc. Assembly, configuration and deployment are well defined in the classical CCM. However, they have to be adapted to specific constraints due to embedability. The underlying execution infrastructure (RT CORBA or others) must be adapted to the constraints of the targets. In Section 5.13 of this Thesis some extensions to IDL for the handling of periodic, aperiodic and sporadic tasks are presented in a preliminary proposal for the future work to be carried in COMPARE. Predefined components For RT/E Interaction models for RT/E component Containers for RT/E Assembly, configuration, deployment, supervision for RT/E Preexisting component Preexisting component container container container Execution Execution Platform Infrastructure Services for RT/E Figure 41: Areas of work for the RT/E component framework 103 Chapter 5 Using CORBA in Control Systems 105 5.1 CORBA Weaknesses Making distributed control is not a simple task. In the bold ICa endeavor to facilitate the construction of such systems, CORBA has been selected as the basic infrastructure middleware to integrate control systems at all levels of the plant (operational, tactical, strategic). But before the technology is deployed the question arises whether CORBA can address the control requirements at all these levels. During this investigation it has been found out that CORBA proves itself worthy in the upper levels of the control pyramid or stack of layers. There are already other technologies capable enough to provide support for applications at these levels but still CORBA provides better support for integration than the rest. The real problems with CORBA begin to show up as we get closer to the lower simple basic loops of control. This is one focus of this Thesis in which extensions to the CORBA model are used to further span its field of application. Autom ation Appli catio ns Infras tructu re Basic S Islan ds o f ervic es Auto m ation Figure 42: A typical automation infrastructure In Figure 42, the typical example of a plant automation layering is shown. In the example, the lower layer is cluttered with what have been called islands of automation. At the beginning of plant automation, islands operate independently of each other frequently using different core technologies to carry out control. It is not a 107 bad situation if all systems in the plant are independent of each other. In the real world, the enterprise can be seen as a whole and no part of it or individual department stands alone without the others. It is just like a composition association of UML entities; their life cycles are tied to each others’. The islands of automation are implemented in most cases by proprietary technology or automation solutions. In order to communicate and to get a common understanding of the enterprise automation as a whole, basic automation services are needed, which are built upon the islands of automation. This basic services layer is where process visualization, alarm and event management, security, device integration (OPC for instance) or messaging services are implemented. As the requirements for this layer are not as stringent as in the islands of automation layer, there is a bunch of technologies that can be used at this level. Until this point, there is not a unified view of the plant. There are only services that allow to interact and to examine the behavior of the different islands of automation. Today’s modern plant automation solutions deploy an infrastructure upon the basic services that allow plant-wide automation applications to be built. This is shown in the last two topmost layers of the figure. Also at this level, specialized CORBA interfaces can be used to provide an infrastructure for plant-wide automation. CORBA integrates horizontally in each layer and vertically in a cross-layer flow of information. As an example of a top automation layer application, the reader might think of Manufacturing Execution Systems. So CORBA is a good fit in all the upper layers of plant automation. In fact, it fits better than other systems because when information crosses layers upwards or downwards, different protocols are tipically used. The lower layer will use some field bus approach to provide the basic level of automation. In the basic services layer the protocol is usually changed to something more mundane as OPC which is used to a great extent. And from here to the upper layers other protocol conversions may be used depending on the types of applications deployed in the plant. Protocols vary from Suite Link to MOM proprietary protocols. The focus here must be put on the lower layer, where resources are low and the requirements for the exchange of information are stringent. By the use of CORBA in all the upper layers of the figure, heterogeneity is resolved by means of CORBA integration. The objective is to integrate also the lowest layer. This layer is always very costly in economic terms due to the use of vendor proprietary protocols and specialized hardware. By integrating CORBA into this layer process information and systems command and control can be made available plant-wide (with the due security authorizations). 108 The case of islands of automation is just an example scenario where CORBA can be used in distributed control. There are other types of distributed systems in which there is not such a hierarchy of layers but in which CORBA can bring numerous development advantages. It is the field of embedded distributed control systems. Examples of this type of systems are brake-by-wire systems, ABS or cabin pressure control systems in aircrafts. In all these systems, distributed specialized hardware and software are used to carry out the control logic and to communicate the devices in a network with hard deadlines. In order to develop core assets for control systems infrastructure to build upon, it is needed that the weaknesses of CORBA regarding control in hard real-time and embedded device conditions be solved. In this case, the infrastructure is a CORBA ORB. Although CORBA was not initially foreseen for industrial systems, it has specifications for real-time, fault-tolerance and embedded systems. Unfortunately, these specifications, specially the real-time specification, miss to treat some of the most important points for control systems. To overcome these difficulties or weaknesses, a completely revamped CORBA ORB has been developed to incorporate functionality for control purposes. Specifically, the following issues in the control domain have been addressed in this Thesis work. • • • • • • • • Sources of overhead or non-determinism in the ORB. Communication models in control systems and in CORBA. CORBA control loops. Event-triggered systems versus time-triggered systems. The notion of time in control systems. Time-stamping. Protocol statistics. Additionally, this Thesis shows the need to address periodic, aperiodic and sporadic activities. These and other issues are being continued within the IST project COMPARE in the context of embedded components. With the items considered, it is possible to have an infrastructure upon which core assets for control systems can be built. It is not possible to have assets with real-time or other control properties if the underlying system, the middleware, does not support them. In this chapter, the modification of an ORB to support these attributes is introduced. Means for deploying CORBA in the lower layers of the automation infrastructure and in embedded systems are presented. Extensions to and profiling of real-time CORBA and other interfaces are introduced that allow building applications in small close-to-process devices. 109 5.2 CORBA Control Systems Domains The collection of domains to be considered in the scope of CORBA Control Systems is the following: • Distributed systems (basically the CORBA Domain): The main characteristic of this type of systems is the distribution of functionality in a collection of components that are running (or can be running) in a collection of processors. CORBA focuses specifically in distributed object systems where the interaction semantics is based on object method invocations. • Distributed real-time systems (the Real-time CORBA Domain): The CORBA base specification does not address topics of real-time systems. The CORBA approach is a best-effort approach where the application developers do not have control on the resources usage to guarantee a real-time behaviour. RT-CORBA addresses these issues providing mechanisms for resource control (processor, communications, memory, etc). • Distributed reliable systems (the Fault Tolerant CORBA Domain): The same can be said of CORBA for development of robust systems. Fault tolerance is not addressed in the basic CORBA specification. The standard for Fault Tolerant CORBA aims to provide robust support for applications that require a high level of reliability, including applications that require more reliability than can be provided by a single backup server. The standard requires that there shall be no single point of failure. • Embedded distributed systems (the Minimum CORBA Domain): For some applications CORBA is too large to meet exacting size and performance requirements. Such scenarios require a cut-down version of CORBA. • Control Systems: Control systems are a domain on its own right. The issues concerning the development of (distributed) control systems are a main concern here. 5.3 The Integrated Control Architecture (ICa) ORB The ICa ORB is the result of a long-term effort towards a solution for distributed control systems that has been extensively funded by several European Union research projects in the framework of the ESPRIT and IST programmes. This ORB originated from the need of reducing the effort employed to develop distributed communications among the processes and devices of distributed control systems in the ICa model. In the late eighties, the obvious solution was to implement 110 an abstraction layer above the details of UDP or TCP/IP programming interfaces to ease the implementation of communications. This was the first attempt to provide some leverage to the problem of communication and gave birth to CCL, the Complex Communication Layer library. This effort was carried out during the works related to the CONEX and HINT projects. In the final eighties and the start of nineties, the OMG began its effort towards an Object Management Architecture (OMA) and its best known specification CORBA. To our research group at UPM, it was clear that an enormous effort was being put in the promotion of the OMA and that the idea of moving to the distributed object world was a promising one. Work began in order to develop an ORB for the industry based upon the ideas of the OMG and in the reutilization of the work done in the CCL library. Basically, the work consisted in the construction of a layer above CCL that allowed us to give system communications an object-oriented approach. An ADL (Agent Definition Language) compiler that was able to generate stubs for interface objects and some multithreading utilities for automatic concurrency support were also developed. The DIXIT project was focused on the development of these tools and its test in the petrochemical plant of Repsol in Tarragona. Although this first ORB was a great advance over the former CCL library as it allowed expressing the system architecture and its interfaces in agent form, it lacked most of the advantages of CORBA. • • • • It followed the philosophy of CORBA but was not fully compliant to the CORBA specification. Thus, all ORB utilities were developed proprietarily. Interoperability was not possible as no IIOP was built into ICa ORB. Interoperability is the primary means to achieve integration and the lack of IIOP precluded ICa from being widely deployed in large complex and heterogeneous systems. In a real-time distributed system, the transmission of quality of service (QoS) properties in the system is essential. ICa did not support this type of nonfunctional information to be transmitted in requests. Heterogeneity also means heterogeneous transports at the network level. ICa did only support TCP/IP or UDP which are not deterministic transports. Also, there was not support for connection management in order to enforce the needed quality of service. All the above circumstances were identified among others, and more funding was obtained to demonstrate that it was possible to incorporate those features efficiently into a distributed-object approach for control systems. Under the umbrella of the DOTS project a completely new ICa ORB was developed, this time considering the 111 CORBA and the real-time CORBA specifications in an approach oriented towards the control world. A full real-time CORBA ORB was developed and extended with additional interfaces and, its fitness for distributed control systems was proved in the scope of SAS (Substation Automation System). The feasibility of embedding the ORB for control in Remote Terminal Units and the interoperability approach with other remote systems running in different platforms and written in other programming languages was demonstrated. This was a major step in ICa as the span of possible applications opened enormously. Although DOTS demonstrated that ICa can be used in traditional control systems, it had not undergone the test of hard real-time. In this Thesis, the effort is put into developing a technology capable of being employed in a span of control applications as wide as possible so there was the need to provide support for such hard requirements. In the case of hard real-time, the problem is the lack of determinism in the ORB. With hard deadlines, a deterministic approach to an ORB is needed. This is not addressed either by current CORBA or real-time CORBA specifications. A new research project, HRTC or Hard Real-Time CORBA, was launched with the objective of identifying the sources of non-determinism in the ORB and to integrate functionality in ICa which allowed running the ORB over deterministic network transports. With these achievements it is possible to use a single ORB or a combination of ORBs (depending of the layer requirements where the ORB runs) at all levels of the control hierarchy and, to integrate heterogeneous systems and networks transparently. 5.4 Levels of Requirements In a real plant or system there will be a varying collection of requirements depending on how close specific parts of the system are to the process. This means that most large real-time systems are multi-rate real-time systems (a form of a sampled data system) where the sampling period for each variable may have different stringency requirements. Multi-rate is just a consequence of the different process variable dynamics and of the different sampling periods needed to achieve smooth input from them. Although multi-rate systems can be single-layer systems, it can be assumed that all systems where there is a hierarchy of layers will have multi-rate sampling. In this case, the different timeliness requirements of all layers are made self-evident. 112 Complex control is the typical case of a multi-rate layered system where output from upper layers is the reference input of several controllers in the lower layers. Although in some cases it is possible for operators to carry out control directly on the lower level controllers (local control), they usually interact with upper layers high-level interfaces. This situation matches the example shown in Figure 42. Layers do not have to be physically linked to each other. There are systems where the layers are linked over the air. Figure 43 shows an example of such a system. Commands Responses Operator-system Interface - From sensors Air-traffic Control State Estimator Virtual Plant Navigation Flight Management State Estimator Virtual Plant CORBA - - Flight Control State Estimator Air Data Physical Plant Figure 43: CORBA air-traffic flight control system (ATC system structure from [LIU 00]) In this CORBA-based air-traffic flight control system there are different timing requirements for CORBA in the different layers of the hierarchy. The system works as follows [Liu 00]. The air-traffic control upper layer manages the flights headed to each destination airport. It develops its mission by assigning a sequence of known geographical points and a time of arrival (metering fix) to every flight. This information is the input for the on-board flight management system in the middle 113 layer. The flight management system calculates a time-dependent flight path to meet the next metering fix at the specified arrival time. The flight management system provides parameters as cruise speed, descend or ascend rates, turn radii, etc. as input to the flight controller in the lower layer and closest to the physical plant. It is easily understood that the periods of computation and response to stimuli in each of these layers is different. The CORBA arrow turns deep red where timing requirements are harder. For the upper layers of the figure, computation time for the control law can be in the order of minutes while for the lowest layer the order of milliseconds must be used. Remember that the lowest layer is a distributed system in itself and that communication of information must be carried out. In order to tackle the requirements of the lower layers of this type of systems, the sources of non-determinism of ICa must be known. By knowing the sources of nondeterminism performance improvements can be achieved and the overall behavior of the ICa ORB can be made more predictable. 5.5 Sources of Non - Determinism ORB sources of non-determinism have been largely studied ([Gokhale 97], [Schmidt 98]). In order to build hard real-time distributed control systems based in CORBA, there is a need to be able to account for the behavior of the ORB regarding the following attributes. • • • • • Determinism and predictability. End-to-end latency and speed. Throughput. Scalability. Footprint. The comportment of an ORB regarding these and other attributes defines its fitness for the control system purpose. Tuning of a real-time ORB is not easy as it is composed of very complex pieces of software that interact and, finding the origin or source for nondeterminism is not always and easy task. Broadly speaking, the source of applicationlevel non-determinism has four main causes. • Lack of Quality of Service interfaces. The CORBA standard does not provide interfaces to control end-to-end QoS. Real-time CORBA partly alleviates this problem by allowing the developer to have control over system resources but still some interfaces are missing. 114 • • • Quality of Service is not enforced by the ORB. Once a client application has delivered the request to the ORB, it must guarantee that the desired QoS for the request is used end-to-end. At the server-side ORB, the request must be properly dispatched according to the QoS and the reply returned to the originating ORB. This is not true for most ORBs. Lack of language and platform support for real-time features. CORBA ORBs are developed using programming languages and run on top of operating systems. This means that although CORBA may specify interfaces regarding end-to-end predictability the underlying infrastructure may not do so. At the end, ad-hoc and non-optimal implementations of critical features are incorporated to ORBs. Lack of ORB performance optimizations. Behavior of the broker is heavily dependant of its architecture and implementation. Thus, the parameters identified at the beginning of the section can be improved by design and implementation of efficient algorithms. This item is a continuous effort as it is known that “software development never ends”. The ICa ORB has been designed with these concepts in mind. In some cases, it is feasible to improve behavior by design (performance optimizations) while in others (language and OS support) it is clearly a matter of providing the best possible implementation with the tools available. Regarding lack of ORB performance optimizations, Figure 44 shows the points in the ORB’s general architecture where improvement is usually needed. Impact on ORB architecture of predictability issues Servant marshalling marhalling Client Skeleton Stub request dispatching marshalling marhalling Memory mgmt ORB POA thread dispatching delay ORB buffering buffering Memory mgmt Figure 44: Points of predictability improvement 115 ICa ORB has been optimized to provide a consistent behavior in all the areas shown in the previous figure. This information is presented in the next section. But predictability and determinism are also improved by control of system resources. System resources control is specified by real-time CORBA and ICa ORB provides an implementation of this specification. Figure 45 shows the possibilities for control of resources that have been implemented in ICa ORB and that conform to the real-time CORBA specification. Provide predictability by controlling ORB behavior Servant Servant Client Skeleton thread priority concurrency Stub connection POA POA protocol ORB ORB Thread thread priority Invoke priority Figure 45: Resource control features in ICa Basically, it is possible to control the QoS at certain levels during the duration of an invocation between client and server objects. This includes controlling the thread priority of an invocation in the client application as well as the management of connections and protocols configuration for the client side. At the server side, QoS is respected and resources are scheduled in accordance with it. Section 5.6 presents the general improvements for performance included in ICa. 5.6 General ICa ORB Performance Improvements As Figure 44 shows, there are several items in a CORBA ORB where optimizations must be worked out. ICa ORB has been built upon efficient implementations of critical code and specialized algorithms have been used to solve involved problems of the ORB. The following items have been addressed. 116 5.6.1 Client Stubs and Server Skeletons In CORBA, the basic mechanism of communication between the application objects and the ORB is resolved by the use of client stubs and server skeletons. Stubs and skeletons are pieces of software that are automatically generated by IDL compilers. CORBA objects are distributed, this means that the object can be used locally as if the object really were in the same host than the client (in fact in modern operating systems it must be in the same address space to obtain a direct request) although it may be many miles away. This ability to make local requests to remote objects makes one of the major advantages of distributed objects; systems programming is kind-of objectoriented. To be able to request a remote object services in an object oriented way, the IDL compiler builds a stub of the remote object. This stub is a shell with the appearance of the remote object that interacts with the client application and the ORB to forward the request to the real server object. At the server side, the ORB delivers the request to the application-implemented object services by means of the server skeleton. The skeleton maps the information received from the ORB to the signature of the operation implementing the service. All this flow of information through stubs and skeletons must be made very efficiently so they must be optimized. In ICa ORB, built in C++, the stubs and skeletons take advantage of the pointer concept to make a very fast implementation. Arguments pointers are identified and paired to their marshalling function pointers and passed onto the ORB in arrays of pointers to carry out further work. Server skeletons are implemented in much the same way but they also support or deal with the burden of helping resolve request dispatching as explained in the appropriate section below. 5.6.2 Marshalling In a CORBA ORB, a network format for data is used to communicate objects that reside in different machine architectures, operating systems or that have been written in different programming languages. Marshalling and de-marshalling take place at the client and server sides. As the process consists in the formatting to the proper representation of the operation to be invoked, the arguments to be passed between client and server, and some operation context information (including QoS information in a real-time ORB), the algorithms employed in the process must be as efficient as possible. The format used in CORBA is called GIOP or General Inter-ORB Protocol. ICa ORB includes a library to format GIOP messages that has been written in an efficient manner. Additionally, specifying an interface among the GIOP and the ORB 117 by means of a library allows code for GIOP to be changed in an easy way, being possible to implement new marshalling strategies in a small amount of time. 5.6.3 Memory Management In a predictable ORB, memory management is an important issue because it affects how the system behaves. Memory is a system resource that must be controlled in order to ensure proper system operation. In the case of embedded systems, memory is a critical matter as the ORB must never go beyond the embedded platform capabilities. The continuous use of dynamic memory (malloc or new way of requesting memory) is expensive and the frequent use of requesting-and-freeing-of-memory cycles leads to heap fragmentation. In many situations, control over memory management is not needed but in the case of embedded systems this kind of resource can be determined at design time, so buffers can be statically dimensioned for this purpose. In ICa ORB, memory buffers can be created with a defined amount of memory so there is control over the maximum amount of memory used for a great part of ORB operations. 5.6.4 Transport Buffering and Delays One of the main sources of non-determinism and lack of predictability is outside the reach of an ORB. It is the transport layer used to communicate the distributed objects. CORBA relies in GIOP and in its implementation over TCP/IP which is called IIOP (Internet Inter-ORB Protocol). Unfortunately, TCP/IP is not a predictable transport and the applications developed over IIOP will suffer from the same problem. To solve this issue, ICa ORB incorporates a pluggable framework architecture based on the Open Communications Interface (OCI) [OMG 01c] and on the Extensible Transport Framework (ETF) specification [OMG 02n]. Both specifications handle similar concepts and their interfaces are based on the same architecture patterns. This framework lets application developers build their own transport plug-ins and to attach them to the framework. It is then possible to create deterministic plug-ins to be used by the ORB. Several transports can be attached to the ORB simultaneously and, the ORB will select the most appropriate one for each request according to the real-time CORBA properties configured for the application. As will be explained throughout this chapter, the transport framework in ICa has been enhanced to support time attributes in its interface. The objective is to have an estimation of the time in which actions progress in the system as needed by hard real-time systems. 118 5.6.5 Thread Dispatching At the client-side, it is easy for the application to control the way thread priorities are handled. In a properly configured ORB, the thread priority of the application will be maintained by the ORB to ensure a uniform quality of service in the path to the object providing the service. At the server side, things begin to get more complicated (servers are CORBA’s most complex part). A server probably manages several types of objects that share resources and can be flooded with concurrent requests at a given point in time. Threads to service requests must be dispatched at the server side as soon as possible and with the due QoS attributes. For maximum efficiency in this process, ICa ORB multithreading architecture uses the reactor pattern [Schmidt 95]. The reactor pattern is also known as dispatcher or notifier and its purpose is to handle requests for a service that are done concurrently by one or several clients. From a software architecture point of view, the reactor pattern allows to employ a simple way of thread dispatching. When software must be developed for a large range of operating systems and devices, like the case of ICa, the use of patterns brings programming simplicity to the software layers that use the pattern. Portability of the software is then improved by the use of pattern solutions. A reactor uses event handlers to service client requests. The reactor normal state is that of waiting on a set of handlers (e.g. a socket file descriptor) by means of a synchronous event demultiplexer (select in the case of UNIX systems). The quality of service attributes associated to the event handler allow to efficiently dispatch an adequate thread when activity is detected for any of the service handlers. 5.6.6 Request Dispatching At the server side there is another problem. Servers can have many objects and each of these objects can provide many services through their available operations. Additionally, the objects live under the management of Portable Object Adapters (POAs or real-time POAs in ICa). The efficient mapping of the service request that comes from the network to the appropriate object operation is an issue that can heavily affect performance depending on the number of active objects and on the number of operations supported by all of them. In order to build scalable applications the performance of this mechanism must be as high as possible. 119 The request demultiplexing or dispatching involves three basic types of object, the ORB, the POAs and the servants. The ORB uses the information of the object key from the Interoperable Object Reference to find the POA that contains the requested servant. Finding the POA can be a time-consuming task as developers may have established a POA hierarchy for the application that must be traversed to search for the right POA. The POA demultiplexes the request to the servant. Multiple servants can be found in a POA. The servant demultiplexes the request to the skeleton that contains the requested operation. The skeleton demarshals the request and executes the associated implementation method for the operation. Notice that a servant may implement different skeletons depending on the POA’s uniqueness policy applied. For real-time and embedded applications it is needed that this demultiplexing mechanism is as efficient and predictable as possible. The reader should think about the consequences for system behaviour when the mechanisms used in each step of the demultiplexing process do not scale well when the number of POAs, servants or skeletons changes. The problem is that CORBA allows a very flexible dynamic behaviour of the architecture. In the critical-systems type of applications, it is recommended that all POAs are created at application initialisation so no adapter activators are used. Servant activation and deactivation is another source of overhead as no static knowledge of servants associated with a POA can be reached. For this reason, it is desirable that no servant managers are used in hard real-time applications. If all POAs are known at application start-up, POA Managers do not introduce additional overhead in predictability of system execution. This is because no child POAs are being created dynamically and work from the POA Manager is reduced to search on a static tree of nodes to control flow of requests. This will put an upper bound to the processing overhead for this layer of the demultiplexing process. To perform request demultiplexing, the search algorithms below are usually employed in most commercial ORBs. When the performance of these algorithms is considered, the O or Φ function notation is used. The O notation is a theoretical measure of the execution of an algorithm, usually the time or memory needed, given the problem size n, which is usually the number of items. For instance, if it is stated that f(n) = O(g(n)) it means that it is less than some constant multiple of g(n). As the most relevant operation for searches is the comparison, this operation is used for the O notation. So if an algorithm needs n comparisons to find a match it is said it is an O(n) algorithm. If it is found in the first comparison it is a O(1) algorithm. To 120 compare the algorithms concern is put in worst-case times as knowing these times allows to provide guaranteed performance predictions. • Linear search: A linear search is done on an array or list checking one item at a time. The average run-time is O(n/2) and the worst-case time is O(n) as n comparisons must be made to find a match. The linear search is also known as sequential search. • Binary search: A binary search is an algorithm to search a sorted array. It begins with an interval covering the whole array. If the search value is less than the item in the middle of the interval, the interval is narrowed to the lower half. Otherwise, it is Figure 46: Comparison of n narrowed to the upper half. Checks are and log n versus n repeated until the value is found or the interval is empty. An interval of n can be divided in half log2n times so a binary search is a O(log n) algorithm. Figure 46 shows that for large numbers a binary search is faster than a sequential search. The problem is that insertion and deletion operations are O(n) algorithms. • Hashing: Hashing is used for the object adapter demultiplexing. It provides O(1) average-time and supports better insertions and deletions than the binary search. The problem is that the worst-case time is O(n), being n the number of duplicates and that may there be a high constant overhead from the hashing function. • Perfect hashing: Perfect hashing is based in a hash function that maps each different key to a distinct integer. Usually, all possible keys must be known beforehand. A hash table that uses a perfect hash function has no collisions. For perfect hashing the number of elements must be known before hand. Perfect hashing executes in constant time and can be compared to addressing an element of an array through its index. O(1) is the worst-case time. To let developers the freedom of implementing POA hierarchies and to associate several servants to every POA a dynamic strategy is needed. ICa ORB demultiplexes incoming client requests through a hash table at the root POA that contains information about POAs and servants associated to a POA. The table is accessed 121 through hash functions. Once the servant has been matched a perfect hash table at the servant is accessed to find the operation’s skeleton. In the prior steps of demultiplexing it is not possible to use a perfect hash functions because POAs and servants are dynamically created. For the demultiplexing of the request to an operation a perfect hash function is used as interface operations are known at compile time. The ICa compiler uses the GNU libg++ gperf() function to generate perfect hash functions for interface operations. gperf() objective is to transform a keyword set W of n elements into a perfect hash function F. F uniquely maps the elements in W onto the range 0..k, where k >= n. If k = n then F is a minimal perfect hash function. gperf() generates a 0..k element static lookup table and a pair of C functions. These functions determine whether a given character string s occurs in W, using at most one probe into the lookup table. 5.7 ICa ORB Specific Features for Hard Real-Time Systems In the previous section, features regarding optimization of the ORB have been presented. Although those features are important for overall ORB behavior we should not miss the point that hard real-time systems demand strong control of system activities. To provide a higher degree of control over the system service, ICa ORB interfaces have been enhanced to deal with aspects of real-time systems that are largely obviated by all ORB implementations. Most of these extensions to a regular real-time CORBA ORB have to do with the management of time. In real-time systems, time represents the hardest part to deal with because it is time the one that gives validity to what occurs in the system. Actions of the system are valid if occur at appropriate instants of time. In ICa ORB, there are hooks in the transport framework that allow an application to be aware of time information regarding requests. There are also IDL extensions designed for the handling of time stamping features and protocol statistics. The proposed extensions for periodic tasks are dealt with in the ongoing COMPARE project. The reminder of this chapter deals with those features in detail. 5.8 CORBA Communication Model Most CORBA systems are based around the client-server model in which a client makes a request and blocks until a reply from the server is received. This is the 122 synchronous two-way communication model commonly used in CORBA. The other most commonly used communication model in CORBA is the one-way communication in which the client does not wait for a reply. A drawback of one-way invocations in CORBA is that, although it is not possible to specify user exceptions for one-way invocations, the ORB is able to throw standard system exceptions which could be raised at the server side and caught at the client side. The invocation control that CORBA can provide should not be considered a drawback in an ideal world. In fact, a connection oriented approach is more advantageous in any type of application as it is possible to manage QoS and to provide isolation in an endto-end manner for the connection flow of data. The problem with the connectionoriented mechanism is that it needs error control to provide reliability and this introduces more delay and jitter. Also, throughput between peer endpoints is more limited as more data needs to be sent. If this is combined with the limited tolerance that many real-time applications show to bad or lost data, the reason why real-time application developers prefer performance before reliability is found. Invocations in real-time applications sometimes must be pure one-way invocations with no error control. ICa ORB is able to deal with pure one-way invocations without error control. With this feature applied, ICa ORB can provide the same behavior at the communications layer as that of any hard real-time system implemented in an ad hoc way. The Messaging specification of CORBA introduced the Asynchronous Method Invocation (AMI) with which operations requests can be made asynchronously using the static invocation interface. The AMI offers two ways of invoking an object operation in a client non-blocking way, being most appealing to control purposes the callback or reply handler model in which the ORB will invoke an application defined reply handler upon reply from the server. This concept of callback is common in event driven systems and has been widely implemented in different types of systems (windowing systems like Microsoft Windows or XWindows for instance). AMI is not part of the ICa broker but its interest for future work resides in the fact that two-way invocations could be made asynchronously. This is a subject for future work. 5.9 CORBA Control Loop This section introduces the basic aspects of distributed networked control for CORBA control systems. As already explained, the lowest layer of the control hierarchy is considered in this chapter. First, the two possibilities for embedding CORBA in this layer are presented then, the main constraints for the timing requirements of the control application are considered. The work for embedding ICa in the lower layer 123 control loop was founded by the European Commission’s IST programme. The project name (IST-2001-37652) was Hard Real-Time CORBA or HRTC [HRTC 02]. 5.9.1 CORBA Controllers In order to build CORBA controllers the role of CORBA in a basic control loop must be determined. Depending on the characteristics of the devices (control and feedback device, sensors and actuators) that form part of the distributed control loop, a CORBA controller can be modeled at least in two different ways: Figure 47: Distributed CORBA Control Loop • Distributed CORBA Loop. This approach is the ORB everywhere approach. In the distributed CORBA approach the sensor node, the controller node, and the actuator node are all equipped with ORBs as shown in Figure 47. Inside the corresponding nodes, the sensor, actuator, and controller are all modeled as CORBA objects. Different possibilities exist with respect to which of these objects should be active objects. Active objects are those considered as proactive objects. They have a goal to achieve and perform actions towards it. The first option is to let the sensor object be an active time-triggered object that periodically takes a sensor measurement and invokes an execute operation on the passive controller object. The execute operation typically is a one-way message. During the execution of the execute operation the controller object invokes the actuate operation on the actuator object passing the control signal 124 as the argument. Also this is a one-way message. Other alternatives include making the controller the active time-triggered object who invokes in every sample the two-way operation getMeasurement on the sensor object, calculates the control signal, and invokes the actuate operation on the actuator object. A drawback of this approach is the risk for sampling jitter in the control loop. Figure 48: Embedded CORBA Control Loop • Encapsulated CORBA Loop. In this approach only the controller node is modeled as a CORBA-compliant object. The controller object implements the communication with the actuator and sensor nodes using non-CORBA technology, e.g. using some field bus technology (Profibus, Modbus, DeviceNet, Genius, etc.). The controller object could be an active object or it could be invoked from some other client representing the task execution control of the controller. The approach is outlined in Figure 48. A similar approach to this one can be found in the Smart Transducers Specification [OMG 03c] that presents a model in which data can be obtained from a network of transducers by means of a CORBA gateway. During the Distributed Object Telecontrol Systems and Networks (DOTS) project which was also funded by the European IST programme (IST-1999-10258), ICa ORB was embedded into a Remote Terminal Unit device for the purpose of building substation automation systems. The model deployed in the demonstration application could fit either alternatives of implementing distributed control by means of CORBA. If the application supports intelligent electronic devices (IEDs) with enough 125 computation power, the model with all nodes CORBA-enabled is used whereas the field bus model is used in the case simple sensors and actuators must be employed. In these CORBA controller architectures, there is always the need of dealing with distributed systems which use a communication link to share information. Since the feedback loop is closed over the network, the stability and control performance is directly affected by the quality of the transport. A network that possesses too long latencies or have too much timing variations may cause the control performance to deteriorate or completely fail. However, if the network characteristics are well known it is sometimes possible to take them into account when designing the control algorithm or to compensate for them on-line. The need for a network with well-known timing properties is evident in the design of such a networked control systems. Both the timing properties of the network transport and of the protocol stacks must be taken into account when analyzing the timely behavior. Means for minimizing the network effect are needed in the CORBA distributed system. For this, a switchable transport framework is needed in the ORB that allows the utilization of predictable network transports where an upper bound for latency and jitter is known. The next two sections dig deeper in the timing constraints that the control loop must confront and into the different delays that occur in the communications path between distributed CORBA objects. 5.9.2 Timing Constraints The basic control loop has two main timing constraints. The first is the sampling period, h, which should be constant, e.g., without jitter. The second constraint involves the input-output latency, τ , from the sampling of the measurement signal to the control signal actuation. This is also known as the control delay or the computational delay. In a distributed control loop the input-output latency also includes the communication delays from the sensor node to the controller node and from the controller node to the actuator node. The constraints are illustrated in Figure 49, where it is assumed that the controller is implemented in a single task. From a control performance point of view the following specifications are important: • The sampling period should be constant, e.g. the sampling jitter should be negligible. This holds for all time-triggered control loops, which is the most common case. An example of the opposite, e.g. an event-triggered control loop is found in combustion engine control systems which normally are sampled against crankshaft revolutions. 126 • • • The input-output latency should be negligible or constant, e.g. without jitter. A negligible latency can be ignored and a constant latency can be compensated for statically in the control design. Most control loops are more sensitive to latency than to sampling jitter. In most cases it is better to have a small latency with jitter than a larger latency without jitter, even if the larger latency is compensated for in the control design. Figure 49: Constraints of loop timing According to the timing constraints, a CORBA ORB like ICa ORB, suitable for distributed control purposes, must provide means to let the application be aware of the progress of network time and to execute periodical tasks in order to provide constant sampling of remote sensor objects and to provide actions on actuators. 5.9.3 Loop Timing As explained before, a good control situation is obtained by equidistant sampling intervals and a negligible or constant control delay from sampling to actuation. However, this can seldom be achieved in practice. Within a node, tasks interfere with each other through pre-emption, and blocking when accessing shared resources. The execution times of the tasks themselves may be data-dependent or may vary due to hardware features such as processor caches. On the distributed level, the communication gives rise to delays that can be more or less deterministic depending on the communication protocol. Some sources of communication delays are: • Processing delay: The time required to process the message (e.g. a packet) within the nodes (hosts, routers, bridges) that the message passes through. At 127 • • • • • the sending and receiving hosts this consists of the time needed to pass through the different protocol layers (link-network-transport), whereas in other hosts it consists of the time it takes to examine the message header in order to decide where to direct the message. The processing delay can also include other factors, e.g. the time needed to check for bit-level errors. Queuing delay: The amount of time that the message spends in the output queue (buffer) of the host or router, waiting to be transmitted. Transmission delay: The amount of time required to transmit all of the bits in the message into the link. For routers and bridges this is also known as the store-and-forward delay. Propagation delay: The time required for the bits to propagate over the link. Transport-level acknowledgement delay: In reliable transport protocols such as TCP, acknowledgments and resending is used to guarantee a reliable connection in spite of bit errors and lost packets. The latter can be caused by buffer overflow due to congestion. This source of delay can be removed if unreliable transport protocols such as UDP can be used. Link-layer resending delay: This is the delay caused by collision detection and the subsequent back-off and resending in multi-access link layer protocols, e.g. Ethernet (CSMA/CD) or CAN (CSMA/CA). This source of delays can be removed if the network is scheduled in such a way that the collisions are eliminated, e.g. using time-division multiplexing or through the separate collision domains of switched Ethernet. While the use of a predictable transport can greatly improve the behavior of the CORBA system regarding the transport layer delays, there are some of these delays which will always be present if the distributed system is too complex. In a heavily distributed control system, it will not be possible to avoid the use bridges and routers and the delays associated to these devices should be accounted for. 5.10 Event Triggered vs. Time Triggered Systems There is a great conceptual difference between time-triggered and event-triggered systems. In a time-triggered architecture, system activities are initiated by the progression of a globally synchronized time base while in an event driven system, activities are driven by other events different than the progression of time. In Section 5.8, the reply handler callback model is a pure event-triggered system as the handler is not activated until the request from the server (the event) has been received. The callback model or the one-way invocation communication model are suitable for realtime applications in which the communication plug-in is event-driven. Unfortunately, this communication model alone is not sufficiently good for time-triggered systems as 128 activities are initiated / synchronized with a global time base 4 . In a system with a timetriggered communication layer, the system activities must be driven from the communications layer and it is the transport plug-in which must be time-aware. An event-triggered system is more flexible than a time-triggered system but the latter is more predictable. In a time-triggered approach all activities and communication take place at predetermined instants of time. In practice, time triggered systems are deployed where the most stringent conditions must be met. Avionics or automotive applications are examples or such systems. Other real-time applications where the system is event-driven can employ transports that are also deterministic but where the system is driven by events. In order not to reduce the scope of applications of ICa, neither of these approaches has been left aside and two specific transport plug-ins for real-time have been developed. A Time Triggered Protocol (TTP) transport plug-in for time-triggered systems and a Real-Time Ethernet transport plug-in for event-driven systems can be used to deploy real-time applications with ICa ORB. 5.11 Distributed Synchronization The problem of distributed system synchronisation is that of ensuring that precedence constraints of jobs on different processors are always satisfied. This problem falls outside the scope of this Thesis. This chapter does address neither multiprocessor / distributed scheduling, distributed resource access control nor distributed system synchronisation. On the contrary, it deals only with real-time optimizations and extensions to CORBA such as those implemented for communications in hard realtime systems. Nevertheless, in a ‘classical’ hard real-time system, tasks will be statically partitioned and assigned to processors and each node will have its own scheduler. In the distributed system, the schedulers need a synchronisation protocol in order to satisfy precedence constraints of tasks’ jobs. The question is what is needed from the real-time communications in order to support distributed scheduling? As mentioned in the next two sections, this depends on the strategy used to synchronise the system. All the nodes in the distributed system are considered to be coordinated at design time in the case of time-triggered applications or to have means at the transport level to ensure that the network schedule holds (in the case of event-driven applications). Actually, it is not needed to be aware of the time of the global time base. Only the difference of time with the time base and the accuracy/precision of the time base in the nodes is needed. 4 129 5.11.1 Clock Driven / Time Triggered Systems In a clock driven or time-triggered approach, to simplify schedulability analysis the system is schedulable only if it is deterministic and all the tasks are periodic 5 . With this assumption, it is possible to transform a loosely coupled system (a distributed system in which it is costly to know global status) into a system that tries to resemble a multiprocessor system (tightly coupled). All the nodes access to data locally which is distributed by the real-time communication system at periodic intervals. The communication network is as if it was a shared memory area and all the nodes schedule their tasks independently of the other nodes. A node knows that certain data will be available locally at pre-specified intervals in time. In this approach, no interfaces for global system synchronisation are required. 5.11.2 RT Ethernet / Event Driven Systems In the event driven approach, CORBA clients issue invocations as in a regular CORBA application but requests may be buffered at the client side to guarantee that the network schedule holds. In this approach, an RT layer is included between the IP and Ethernet layers. The RT layer is responsible for the traffic control for worst-case scheduling. The RT layer queries a net guard node which schedules real-time traffic in the network. This approach does not need additional interfaces for synchronisation. With the constraints exposed in the previous two points, nodes act as if they were independent nodes and system schedulability depends only of the schedule of each node. System nodes are not coupled to each other. Anyway, the schedulability of the whole system is application bound and is not addressed in this Thesis. 5.12 The Notion of Time In a real-time system actions or results must be accomplished before the deadlines of the system that depend on the environment are missed. The system is called a hard real-time system in the case catastrophic results are expected if a deadline is missed. Flight control, brake and stability control or drive-by-wire are common examples of such systems. Modern operating systems do not deal with the concept of time to express tasks priorities because the complexity of the operating system and of the underlying hardware (e.g. several levels of processor caches) makes it almost impossible to forecast when a task will be finished. Anyhow, if the ICa ORB is to be Provision can be made for sporadic tasks to be accomodated in the off-line scheduling of the system but this does not fit naturally into a time-triggered system. 5 130 embedded for hard real-time applications it is needed to let the application deal with the progression of time. This is achieved by building in ICa ORB the concept of timeaware communication protocol plug-ins. 5.12.1 Driving the System from the Communications Layer For hard real-time applications the ORB must be aware of the progression of time. This must hold for the distributed system. With this requisite and for a time-aware network protocol like the Time-Triggered Protocol, time data should be obtained from the communication layer and made available to the application or the ORB. This can be achieved by extending current definitions for the Open Communications Interface or the Extensible Transport Framework submission. In ICa ORB, the Open Communications Interface has been extended to allow the retrieval of temporal information. Nevertheless, in this document the extensions are presented for ETF as it is the adopted transport plug-in specification by the OMG. OCI and ETF are extremely similar and basically all the actors involved in one specification exist in the other so translation from one specification to the other is a rather simple exercise. A pluggable transport must be able to tell the application what time it is as well as a measure of time precision and accuracy. In the case of periodic applications, it is also interesting for the application to be signaled about the beginning of the next period or cycle of execution. Proposals for the handling of periodic, aperiodic or sporadic tasks are made later in this chapter. This kind of service is proposed as future work and will be addressed in the COMPARE project. It must be noticed that time-awareness for the ORB or the application might be less precise than that of the transport protocol being then necessary to downgrade the time for the application. 5.12.2 Interface Extensions to the Extensible Transport Framework In order to either drive the ICa ORB or the application activities from the communications layer, it is needed to incorporate the notion of time into the pluggable protocol framework of ICa. The representation of time becomes then a crucial point as it should be simple and consume as less processing power as possible. In the traversal approach adopted in this work, advantage of the definitions already made in the OMG document Formal/03-01-01 Smart Transducer Specification [OMG 03c] is taken. In this specification a time instant is represented in IDL as 131 typedef long long TimeInstant; and its data representation is the following (excerpt from the specification): “This type is used for timestamps. The 40 upper bits represent the number of seconds (after 34841 years an overflow will occur) while the remaining 24 bits represent the fractions of a second, allowing an accuracy of 60 ns. In a system with external clock synchronization the 40 upper bits are initialized with the value 0 at 00:00:00 UTC on January 6, 1980, which is also the reference starting point (the epoch) for GPS-time. In this way every point in time 17420 years before and 17420 years after January 6, 1980 can be uniquely represented with an accuracy window of 60 ns. Stand-alone systems without external clock synchronization are set to 0 during initialization.” [OMG 03c] also defines a time duration type typedef long long TimeDuration “This type is used for durations that are represented in units of 2 60 ns).” -24 as: seconds (about There is also a representation of an instant of time: struct Instants { TimeInstant instant; TimeDuration period; }; and its purpose is the following “The first value (subfield instant) informs about the next instant when the most recent of the denoted events will occur. The second value (subfield period) is the period of the named data item.” This representation of time can be used in the pluggable protocol framework to add a service that allows the real-time system / ORB to learn when the next cycle of processing / execution (read / write) must begin. 5.12.3 Where to Ask for the Time in the Pluggable Framework? The pluggable transport is the entity in the architecture of ICa that is aware of global time. The pluggable / extensible transport framework provides means to forward the time information way up to the ORB. Notice that forwarding the time to the ORB does 132 not mean that the real-time application is time-aware. The communication plug-in interfaces are not available at the real-time CORBA application level. Then, it is also necessary to provide an interface for the application to gain knowledge of the progression of time. Regarding the Extensible Protocol Framework or the Open Communications Interface, a pluggable protocol framework handles the concepts of registry, factory, connector, transport and acceptor (see Figure 50). Only the transport is used when the system is working to communicate data. The reminding entities are used for configuration and setup of connections. Once a connection has been established, it is the transport object the one in charge of communications. This means the interface related to the transport either in the OCI or in the ETF submission should be extended to let the ORB ask about the progression of time. It is not possible to ask for the time without being connected to a transport. CLIENT SIDE IMPLEMENTATION SERVER SIDE TCP/IP Conn. Fact. TCP/IP Connector TCP/IP Transport TCP/IP Acceptor TCP/IP Acc. Fact. Conn. Fact. Connector Transport Acceptor Acc. Fact. ABSTRACT n n 1 1 Conn. Fact. Reg. creates Acc. Fact. Reg. Application (ORB) Figure 50: The ICa OCI transport plug-in architecture Figure 51 shows the layered architecture of the real-time ICa ORB with detail of the extensible transport framework and two transport plug-ins attached (TTP and RTEthernet transports). The time arrow shows how propagation of time values occurs in the ORB and in the CORBA application. In this approach, the global progression of time is only known by the transport layer (e.g. the TTP protocol) and the extensible 133 protocol framework interface provides means to allow the ORB to ask for the time. The figure also shows that it is the message layer of the ICa ORB (the GIOP layer) the one with complete access to the plug-in interface (e.g. to the transport object) so without further interfaces it is not possible for the CORBA application to access directly to the transport to learn the time. Stubs/Skels Time (RT)ORB ORB Transport Layer GIOP OCI/ETF RT Eth. TTP Vendor independent interface Network Transport Layer Figure 51: Source of time information in ICa 5.12.4 Particularization for the ETF Submission To ask for the time in the ETF submission, the interface which defines the role of the transport must be identified. In the ETF, this role is represented by the Connection interface. The Connection interface also serves as a connector and is able to establish connections to servers. The following IDL code is the ETF Connection interface extended with operations to ask the global time as seen by the communications protocol. module ETF{ // // declaration of time types // typedef long long ProtocolTime; typedef long long ProtocolDuration; struct ProtocolInstant{ ProtocolTime instant; ProtocolDuration period; // // // // // a time instant a duration current time and period time event period of time event 134 octet precision; // precision }; // locality constrained local interface Connection { void write(in boolean isFirst, in boolean isLast, in Buffer data, in unsigned long offset, in unsigned long length, in TimeBase::TimeT time_out); void read( inout Buffer data, in unsigned long offset, in unsigned long min_length, in unsigned long max_length, in TimeBase::TimeT time_out); // transport needs to set data.length() to // offset + number of bytes actually read void flush(); void connect(in Profile server_profile, in TimeBase::TimeT time_out); void close(); boolean is_connected(); Profile get_server_profile(); boolean is_data_available(); boolean wait_next_data(in TimeBase::TimeT time_out); readonly attribute long id; readonly attribute boolean supports_callback; readonly attribute boolean use_handle_time_out; // protocol time readonly attribute ProtocolInstant protocol_time; // next available transmission time readonly attribute ProtocolInstant next_transmission_time; // time needed for the delivery ProtocolInstant delivery_time( long size ); //in bytes }; }; In this IDL, time is represented as in the Smart Transducers Specification. This representation of time allows to easily synchronize a site with a granularity of up to 60 ns, e.g. with the signal from a GPS receiver (since for this time-format the epoch of GPS-time has been chosen). The transport will provide information about the achievable precision in the particular node. 135 For a cluster (e.g. a cluster of TTP nodes 6 ) running on its own it is not necessary to compare the timestamps with other timestamps outside the cluster. In this case it is not necessary to provide an absolute time base to the system. Only the differences of time between node clocks are needed. The ProtocolTime and ProtocolDuration have the same interpretation as in the Smart Transducers Specification. The same holds for the precision octet which represents the number of significant bits in the timestamp (this parameter does not reflect the accuracy between the clocks in the system and a reference clock but just the precision that is achieved among the clocks in the system). Time Precision is (from [OMG 03c]): “The Precision represents the number of significant bits in the timestamp. This concludes in an error window of 2 39-PREC seconds. Valid values are from 0 (no precision; the timestamp might be a random value) to 63 (an error window of about 60 nanoseconds). Note that this parameter refers to precision within an ST system, not to the accuracy between the clocks within an ST system and the external time reference.” The ProtocolInstant structure is able to provide information for the current time, a period of time and the precision of the time measure. With this data type, it is possible to add an attribute and an operation to the Connection interface. The protocol_time attribute represents the global time and its precision as seen from the communication protocol. The period field is unused in this case. For the next_transmission_time operation, the pluggable framework must obtain timing information regarding the next protocol instant in which messages can be sent from this node, the instant data member of the struct is the time instant in which the next message transmission will begin, precision is the precision of this measure and period is the period of update. Knowing the period of update, ICa clients can be synchronised to make requests at proper times. The third operation, delivery_time, obtains information about the needed time by the transport to deliver a certain amount of data expressed in bytes (size parameter). A cluster of TTP nodes is a set of computing nodes that share a physical TTP infrastructure and are statically scheduled to provide the temporal guarantees that the time triggered protocol provides. 6 136 The operation returns a ProtocolInstant structure in which the instant member is the next instant in which it is possible to send a message, the period data member contains information about the needed time to send the amount of data and precision is the precision of the time instant. 5.12.5 Asking the Time from an ICa Application With the extensions for the Extensible Transport Framework it is possible for the ORB to request to the underlying communications layer the protocol time and also to inquire when the next message slot will occur. Also it is possible to know the amount of time needed to transmit a certain quantity of data. But the ORB is aware of neither the purpose nor the type of messages handled by the application on top of it. For efficiency reasons, the ORB cannot be laden with the task of actively informing the application about timing information (a different thing is that of signalling periodic activities). This task should be initiated by the application. In real-time CORBA, the CORBA::Object can be configured to use a certain set of communications protocols. This is called protocol configuration in the real-time CORBA specification. At the client-side of the application, the policies for protocol configuration can be applied at the object level. This means that it is possible to establish a relationship between instances of protocols and objects in the client side of a real-time CORBA application. It is then necessary to extend the object interface to let the application ask for the time at the object level. As the field of application for timing operations is vertical (only specialised applications of a domain will make use of the interface) rather than horizontal it is not a good idea to extend the CORBA::Object interface. Instead of this, it is better to extend the functionality of the object in the RTCORBA module so non real-time applications do not have the additional operations regarding handling of time. Using pseudo-IDL the object interface is extended in real-time ICa as follows: module RTCORBA{ local interface RTObject{ // protocol time readonly attribute ETF::ProtocolInstant protocol_time; readonly attribute ETF::ProtocolInstant next_transmission_time; ETF::ProtocolInstant delivery_time( long size );// size in bytes }; }; 137 In the case of the RTObject, the time type appears as ETF::ProtocolInstant. As the ETF also depends on Real-Time CORBA for the declaration of protocol policies it is better if the ProtocolInstant type is defined in the RTCORBA module. In the case of the ETF or the OCI, the declaration of the time types must be removed from the IDL specification and an #include “RTCORBA.idl” directive is used at the beginning of the file. The IDL follows for the RTCORBA module: module RTCORBA{ // declaration of time types long long ProtocolTime; // long long ProtocolDuration; // struct ProtocolInstant{ // ProtocolTime instant; // ProtocolDuration period; octet precision; // }; a time instant a duration current time and period time of event // period of the event precision local interface RTObject{ // protocol time readonly attribute ProtocolInstant protocol_time; readonly attribute ProtocolInstant next_transmission_time; ProtocolInstant delivery_time( long size ); //in bytes }; }; The RTCORBA::RTObject interface is not derived from the CORBA::Object interface as we are using pseudo-IDL for which inheritance is not defined. Nevertheless, RTObject is conceptually an extension of the CORBA::Object interface. There is a single instance of RTCORBA::RTObject per instance of CORBA::Object. Notice that the interface extension has been declared to be local. The extension of the interface adds functionality only for handling of time purposes and this information shall only be valid for a given node so ICa will throw an exception if an attempt to marshal an RT object is made. 5.12.6 Life as a RTObject Narrowing to an RTObject is not free; there are some rules to follow when an object reference is a reference to a RTObject. • • • RTObject instances are client-side objects. All CORBA::Object instances may become RTObject instances. A RTObject instance cannot be passed as a parameter of an IDL operation nor can it be stringified. Any attempt to do so shall return in a MARSHAL 138 • system exception with a Standard Minor Exception Code of 4 (attempt to marshal a local object). A RTObject narrowed to other narrower types cannot be passed as a parameter of an IDL operation nor can it be stringified. Any attempt to do so shall return in a MARSHAL system exception with a Standard Minor Exception Code of 4 (attempt to marshal a local object). If a RTObject is widened to a CORBA::Object, it is no longer a RTObject. Making a RTObject a local object helps resolve the problem of implementing _narrow() and _is_a(). Remember that all CORBA clients are based in the CORBA::Object interface. Although a client object can be represented as a RTObject, the server side will only be an Object. However, this artifact allows to know the time for the pair object / protocol without modifying the existing CORBA specification. 5.13 Periodic, Aperiodic and Sporadic Activities This section has been included here as it fits into the discussion of this chapter. The contents of this section are a proposal for the future extension (see Section 4.11.3, the COMPARE project) of ICa ORB in the way activities of a real-time systems can be handled. In a hard real-time system we can find three types of tasks: • • • Periodic tasks. Sporadic tasks. Aperiodic tasks. In the scope of hard real-time CORBA a task is identified by a CORBA request which in turn can start several jobs in the system to carry out a system function. It is necessary for the ORB to identify which tasks are periodic, sporadic or aperiodic. In this distinction, we assume that it is known at design time which tasks correspond to each of these types. The distinction of tasks is provided as an IDL extension in the signature of an interface operation so it can be interpreted by the IDL compiler and information about the nature of the task can be built into the stubs and skeletons code by the IDL compiler. Operation declarations in OMG IDL are similar to C function declarations. The syntax of an operation declaration is shown next. <op_dcl> ::= 139 [ <op_attribute> ] <op_type_spec> <identifier> <parameter_dcls> [ <raises_expr> ] [ <context_expr> ] <op_type_spec> ::= <param_type_spec> | “void” The optional operation attribute indicates which invocation semantics the communication system should provide when the operation is invoked. The use of an operation attribute is optional. The syntax in regular IDL is the following: <op_attribute> ::= “oneway” The one-way attribute indicates that the operation semantics are best effort, with no delivery guarantee of the call. One-way semantics also indicates that the operation is invoked at most once. In the proposed ICa IDL extension, the operation attribute is extended to provide information about the nature of the interface operation: <op_attribute> ::= [<op_model>] [“oneway”] <op_model> is an optional operation model which is defined as: <op_model>::=”periodic”<T>”,”<D> | “aperiodic” <D> | “sporadic”<d>”,”<D> | Aperiodic tasks can occur in bursts and its worst execution time cannot be known before hand. We have called sporadic tasks to those that at least are interleaved by a delay d. In the case of periodic / aperiodic tasks, D is the hard / soft deadline of the tasks while for sporadic tasks it is hard deadline. T is the period for periodic tasks. T, D and d time units depend on broker implementation. It is responsibility of the ORB to invoke periodic tasks (once the first invocation has been made) and to schedule aperiodic and sporadic tasks 7 . For periodic tasks, it is useful to declare arguments by reference. A new modifier for parameters has been devised for this. It has been called ref that means that when the ORB periodically executes the periodic task the parameter value will be actualised. For example, in C++ it can be achieved by using references to objects as parameters. As an example, suppose there is an interface temperature_actuator which has an operation to periodically set the temperature value to a certain value each 30 ms. The signature of the operation can be expressed: This means a lot of work will have been done at design time to schedule the system. For instance, to verify that the TDMA network schedule will provide room for sporadic tasks. 7 140 interface temp_actuator{ periodic 30,10 oneway set_temperature(ref float temp); }; The previous example corresponds to a periodic activity of the system. That the activity is periodic means it has a known release time and that it has a hard deadline (also that is executed with a known interval). The operation model attribute allows the IDL compiler to generate stubs where this information is available to the ICa ORB. In the case the optional model attribute is missing, it will be understood that the operation is aperiodic. Aperiodic operations can be considered as regular CORBA operations. When the client invokes an operation, it is passed on to the ORB which in turn invokes the transport plug-in to carry out the invocation. The operation model attribute will be passed to the ORB along with the GIOP request. This way the ORB has the possibility of scheduling the request taking into account the operation model. The operation model also carries information regarding release time and deadline of sporadic tasks. This is information is treated by the ORB in the same way as periodic tasks. In the case of periodic operations, the scheduler will be prepared to handle the request 8 but in the case of aperiodic or sporadic operations the scheduler may perform some acceptance test before delivering the invocation. If as a consequence of performing the acceptance test an invocation is rejected the system will raise the standard system exception NO_RESOURCES. 5.14 Timestamping A RTObject can be requested to timestamp replies to requests on arrival. This allows the application to have not only value information but timing information of the flow of requests. The instant of arrival of a reply to a request can be known in ICa ORB. A RTObject must be instructed to timestamp replies to requests so the overhead of timestamping can be avoided in the ORB if the application is not handling this type of functionality. A TimestampPolicy policy object has been incorporated in the RTORB of ICa to support timestamping of requests. In most hard real-time systems the periodic operations schedule is determined at design time. 8 141 module RTCORBA{ local interface TimestampPolicy:CORBA::Policy{ }; interface RTORB{ TimestampPolicy create_timestamp_policy(); }; }; The TimestampPolicy can be used to configure a client-side real-time object via the CORBA::Object::set_policy_overrides operation. The policy is applied only at the client side. After setting the policy subsequent invocations by the client object will be timestamped by the ICa ORB. An object has access only to the timestamp of the last invocation. The functionality is supplied as an operation of ICa’s RTObject interface. module RTCORBA{ // declaration of time types long long ProtocolTime; // long long ProtocolDuration; // struct ProtocolInstant{ // ProtocolTime instant; ProtocolDuration period; octet precision; // }; a time instant a duration current time and period precision local interface RTObject{ // protocol time readonly attribute ProtocolInstant protocol_time; … // time stamping interface ProtocolInstant time_stamp(); }; }; The timestamp operation shall be called just after an invocation has been issued. Attempts to invoke time_stamp without setting the policy for the object shall result in an INV_POLICY system exception with MINOR_CODE of 1. 5.15 HRTC Protocol Statistics Protocol properties as specified by real-time CORBA can be described in IDL for each specific protocol. They are not defined here as it is foreseen in real-time CORBA to provide those properties for each different protocol implementation. The protocol properties are devised to dynamically configure a particular transport. 142 However, it has been found that in most real-time systems (mainly in event-driven systems) a useful tool is to provide means in the ORB to gain knowledge about the performance of the transport object. In ICa ORB, this functionality has been gathered under the concept of protocol statistics. An interface for protocol statistics has been added to the ORB. The HRTCProtocolStatistics interface contains transport data which may be relevant for real-time applications. The interface has been defined to be a local interface as it should not be marshalled outside the local process scope. The definition of this interface follows. local interface HRTCProtocolStatistics{ // minimum delay in microseconds readonly attribute long min_delay; // average delay in microseconds attribute long avrg_delay; // maximum delay in microseconds attribute long max_delay; // packet loss as probability attribute double packet_loss; }; To be able to take advantage of the statistics, the ORB must provide a handle for the application to request the information. The ICa real-time ORB interface has been enhanced with an operation to get the statistics from a particular transport. The transport is identified by its protocol tag as several transports might be attached to an object at runtime. The operation signature is shown below. interface RTORB { HRTCProtocolStatistics get_protocol_statistics( in ULong protocolTag ); }; 5.16 C++ Source Code Example This section shows an example of how a client-side ICa application can be configured to use a transport protocol and how timing information can be obtained at the application level. Most of the code is start-up code and is written only once as bootstrapping code for the CORBA application. The excerpt of code shows how the client-side application is able to ask for the time associated to a specific network protocol. // Build factory object for the new transport //(inherited from OCI factory classes) // only connector is needed at the client side 143 HRTCConFactory ConFactory _HRTC; // Get a reference to the RTORB CORBA::Object_var obj = orb -> resolve_initial_references( "RTORB" ); RTCORBA::RTORB_var rtorb = RTCORBA::RTORB::_narrow( obj ); // Get a reference to the OCI ConFactoryRegistry CORBA::Object_var obj2 = orb -> resolve_initial_references( "OCIConFactoryRegistry" ); OCI::ConFactoryRegistry_var ConFactReg = OCI::ConFactoryRegistry::_narrow( obj2 ); // Add an acceptor factory object ConFactReg ->add_factory(ConFactory _HRTC); // Get a reference to the Naming Service CORBA::Object_ptr nm = rtorb-> resolve_initial_references("NameService"); CosNaming::NamingContext_var nc = CosNaming::NamingContext::_narrow( nm ); // Resolve the object by its name CosNaming::Name name_rtObj; name_rtObj.length(1); name_rtObj [0].id = CORBA::string_dup( "RT_OBJECT" ); name_rtObj [0].kind = CORBA::string_dup( "" ); CORBA::Object_var object = nc ->resolve(name_rtObj ); RTCORBA::RTObject rt_object = RTCORBA::RTObject::_narrow(object); RTObjectTest_var rt_test_obj = RTObjectTest::_narrow( rt_object ); // Set an object protocol configuration policy override HRTCProtocolProperties props; // assigned by default RTCORBA::ProtocolList Prot_list; Prot_list.length(0); Prot_list[0].protocol_type = TAG_HRTC; Prot_list[0].transport_protocol_properties = props; // Prot_list[0].orb_protocol_properties – defaults to GIOP RTCORBA::ClientProtocolPolicy_var clPol = rtorb-> create_client_protocol_policy(Prot_list); // Assign the protocol policy to an object // after this the object uses the HRTC protocol rt_test_obj->set_policy_overrides( clPol ); // To ask for the time a connection must be validated PolicyList_var incosistent_policies; rt_test_obj->validate_connection(inconsistent_policies); 144 // Get the time RTCORBA::ProtocolInstant_var prot_time; Prot_time = rt_test_obj->protocol_time(); 5.17 The TTP Transport Plug-in This section presents the options that were considered for the design of a TTP transport over the OCI or the Extensible Transport Framework in order to have TTP communications in a HRT CORBA broker (ICa ORB). TTP and its target hardware impose some constraints on the way a transport plug-in for a hard real-time CORBA broker can be implemented. The underlying TTP technology this transport uses has been developed by Professor Hermann Kopetz’s group at the Vienna University of Technology. The TTP transport plug-in was developed during the works of the IST2001-37652 HRTC project and its design and implementation as a hard real-time protocol plug-in is part of another PhD Thesis work in our research group. 5.17.1 Main Constraint for a HRTC TTP Transport Plug-in The TTP working mechanism consists of periodically exchanging state values between Communications Network Interfaces (CNI). Then, the main feature to consider for TTP regarding the implementation of a transport plug-in is that state values are periodically copied from one endpoint’s memory to other endpoints’ memory buffers. The period of the copying mechanism is predetermined beforehand at design time. With this consideration, there are four options available in order to develop a transport plug-in for ICa. 1. 2. 3. 4. GIOP messages written into CNI. Pluggable data-interface. Active server. Standard CORBA + TTP/C application. Advantages and disadvantages of either option are explained in the following sections. 5.17.2 GIOP Messages Written into CNI TTP periodically exchanges state values between CNI memory endpoints. It is possible to consider a GIOP message as such a state value. This way, a GIOP message will be forwarded by CORBA clients to the transport plug-in. The transport plug-in 145 will be responsible for updating the CNI memory with the new message. During the dedicated time slot of the node in the TDMA schedule, the GIOP message will be copied to the other TTP nodes CNI memories. The transport plug-in at the server side is notified of the arrival of the message and the GIOP message is forwarded to the server endpoint ORB. At this moment, the server ORB is able to process the GIOP message and to deliver the request to the appropriate servant. This mechanism is shown in Figure 52. Notice that, as TTP periodically copies the GIOP message between client and server endpoints, clients only need to make one request to trigger a “get state value” mechanism. There is no need to make a new request if the contents of the request have not changed. This means that in TTP requests can be executed periodically. CLIENT CLIENT ORB ORB GIOP GIOP OCI OCI CNI CNI GIOP GIOP Figure 52: GIOP messages copied directly to CNI memory ADVANTAGES This implementation of the transport plug-in has the following advantages. • • • • The implementation is generic. It is independent of the application as full GIOP messages are exchanged between the endpoints. The modification of the broker implementation only affects the development of a new OCI or ETF transport plug-in. No changes at the ORB are required. It is possible to pass policies, service contexts, etc. together with the request as in a regular CORBA or Real-Time CORBA ORB. Servers can be configured from the client-side. This makes this option more “CORBA compliant”. Less implementation efforts compared to the other options. 146 DISADVANTAGES The disadvantages of this approach are: • • • 5.17.3 At each server a dedicated CNI memory slot is needed for each client (unicast communication relationships). This is a limitation of the TTP cluster hardware architecture. Limited CNI memory (2 Kbytes) might require fragmentation of messages (TTP packet service). Every client needs to issue an initial request. Pluggable Data Interface The second option has been named after the pluggable data-interface. In this option, the TTP communication mechanism is preserved in the sense that only state values are exchanged between client and server CNI memory endpoints. In this case, the implementation of the transport plug-in on the client side must preprocess the GIOP message and write into the CNI memory only the value to be exchanged. At the server side, the value from the CNI memory must be encoded into a GIOP message to be passed to the ORB. This is necessary in order to maintain ORB interoperability. It is possible to match a value with a CORBA request on the server side as in TTP each value is written / read in a predefined memory position. Hence a value can be matched to a request. This approach is depicted in Figure 53. CLIENT SERVER OCI GIOP OCI GIOP CNI VALUE CNI VALUE VALUE = f(GIOP) GIOP = f(VALUE) Figure 53: Pluggable data interface ADVANTAGES The advantages of the pluggable data-interface are the following. 147 • • • Only the transport plug-in must be modified. No changes are necessary for the ORB. Resources in the TTP hardware are scarce (mainly the CNI memory). This option has the advantage that it is more memory efficient because only values are written into the CNI memory. No initial client request is required to trigger the exchange of data. The plugin issues server requests automatically. Clients can access state values locally when required. DISADVANTAGES The pluggable data-interface also has some disadvantages. The most important ones are described below: • • • • 5.17.4 This solution is application dependent. A new plug-in must be developed for every TTP application. This is due to the fact that the plug-in must know that a certain area of the CNI memory is used for a specific CORBA request so it is able to build a GIOP request and pass it to the broker in order to invoke the code of the servant at the server side. Less CORBA compliant. Only state values are exchanged between client and server endpoints. It is not possible to specify policies at the client side to be applied at the server side or the other way round. This option requires more computational resources to parse GIOP in the transport plug-in. There is more processing overhead as GIOP messages are processed twice. Currently, the GIOP layer cannot be by-passed in ICa. Performance depends of the relationship between host speed and network bandwidth. Active Server In CORBA, objects are passive entities that react only upon requests from clients. In hard real-time applications, sampling of values (e.g. a temperature sensor) is commonly done on a periodic basis. This means that values must be available at the server side for clients to grab them. In a TTP application, state values can be periodically copied from the server to the client endpoints so clients only need to locally read the value to get the last sample. In this model, the transport plug-in at the server side is responsible for building the request and passing it to the ORB at predefined instants to receive back from the ORB the new value for the sampled data. 148 In this model the application at the server side is driven from the communication layer and the server code is initiated at predetermined instants of the global time. It is possible to think that in this approach the communication system is a client of a passive server. ADVANTAGES The advantages of this approach are summarized as follows. • • • Can be combined with the “GIOP message written into CNI” or with the “pluggable data-interface” options. This option supports the notion of global state variables as server values are spread by TTP across the system and are learnt locally by the client objects. No GIOP message-processing overhead exists as the ORB and the IDL compiler are specialized for this type of transport. DISADVANTAGES As happens with the other options, active servers have also some disadvantages. • • • 5.17.5 Requires changes to OCI, ORB and probably to the IDL compiler. It is the more complex option and requires larger effort to implement. ORB interoperability is lost for this type of plug-in (no GIOP messages). Standard CORBA + TTP/C Application This approach uses CORBA for diagnosis and maintenance purposes. The exchange of real-time data is performed by the TTP/C framework in an exclusively reserved part of the respective sending slots. In a prototype, access to the real-time data is granted by application specific code that works independently from CORBA. It is possible to automatically generate code that performs this kind of operation without additional programming effort from developers. ADVANTAGES The advantages of this approach are summarized as follows. 149 • • optimal temporal behavior fully compatible with CORBA DISADVANTAGES The main disadvantage of this approach is that • 5.17.6 The real-time part is not CORBA-based although it would be possible to support this solution by a CORBA framework with code generators for the specific TTP/C application. Implemented Option for TTP/C The Active Server is the best solution from a technical point of view, since it can be built in an application-independent way. Further, the inner working of the solution provides an acceptable performance. The great drawback of the solution is that it requires a vast amount of changes to the ORB and the IDL compiler and that a main premise of CORBA is lost, the ORB interoperability. At the design and implementation stages, it was decided to have a pragmatic approach. What is needed to prove is that a TTP transport makes the ORB more deterministic, that it is possible to improve end-to-end predictability. In this approach, it does not matter if a request takes a few microseconds more or less to be transmitted between endpoints if the delay and jitter are always the same. Of course, in the real-world it will be important depending of the type application. With these premises in mind, the “GIOP Messages Written into CNI” approach was the first attempt to be implemented over the TTP hardware. Due to the limited CNI memory and the quite big chunks of information per data-value (message encapsulation), this solution was not able to exhibit the optimal performance of the hardware regarding speed but, the results where acceptable for a wide range of systems. Additionally, this approach is an intermediate result for the implementation of the “Pluggable Data Interface”. In the prototype, the application dependency is solved by hand-written code, although in a production environment it is possible to support the programmer with a tool that generates most of the application dependent code from an abstract description of the application interface. 150 5.18 The RT-Ethernet Transport Plug-in This transport uses worst case scheduling technology from Professor Karl-Erik Årzén’s Group at the Department of Automatic Control of Lund University, Sweden. It is one of the world’s leading groups in control theory and engineering and the plugin was developed in the works of the IST-2001-37652 HRTC project. The RTEthernet plug-in is a solution for real-time communications using a switched scheduled Ethernet (RT Ethernet) as a pluggable transport layer within the OCI or the Extensible Transport Framework. The main advantages and features of this approach in comparison with standard CORBA (IIOP) and CORBA on top of TTP are the following: • • • • • • The approach is based on switched Ethernet unicast communication between a sender (CORBA client) and a receiver (CORBA server). This is in contrast to TTP which is based on multicast. Unicast communication has both advantages and disadvantages compared to multicast in a hard real-time setting. A large advantage is the better scalability and resource utilization for large applications. Unicast communication is also the basis for standard CORBA. A disadvantage is that fault-tolerance is not built-in. The communication approach is invocation-oriented, in the same way as in standard CORBA. A client that wants to send a hard-real time message to a sender does this by making a remote invocation on an operation of a CORBA object on the server side. The communication is event-triggered rather than time-triggered. A GIOP request initiated by the client corresponds to the event that triggers the communication. However, in order to guarantee that the network schedule holds, GIOP requests may be buffered and delayed at the client side by a realtime layer inserted in the protocol stack. In the plug-in, only asynchronous one-way invocations are supported. Replies to invocations are treated as separate one-way invocations in the opposite direction. However, in principle it could be possible to also handle real-time two-way invocations. The RT Ethernet approach allows hard real-time communication in parallel with ordinary “soft” CORBA/IIOP communication using the same transport. By scheduling the access to the network (bandwidth-limited communication) it is possible to obtain hard worst-case bounds on the communication latency. Using the standard definition of hard real-time used within the real-time community, e.g. guaranteeing that deadlines are met, the RT Ethernet approach is a hard real-time communication approach. However, the jitter latency can be considerably larger than with the TTP approach. 151 • • • The approach does not require any global clock. However, if a global clock is available the schedulability may be increased and the latency jitter decreased if the transmissions are scheduled based on the global clock. Due to the lower degree of temporal determinism compared to TTP, the approach is less suited for safety-critical hard real-time applications. However, a very large amount of hard real-time applications are not safety-critical. Some examples are industrial robotics and automotive engine control. The approach is based on low-cost, high-speed COTS technology. No special hardware is required. Using RT Ethernet for hard real-time communication involves two activities: 1. Initialization of the communication. This can be considered to be done without any hard real-time requirements. 2. The actual communication. This is performed in real-time. This document primarily devotes to describe the real-time communication, to a large degree omitting the way the communication information has been initialized and set up. 5.18.1 Switched Scheduled Ethernet The RT Ethernet transport technique is based on switched full-duplex fast (100 MBit) Ethernet. Every node in the network is connected directly to the switch. The advantages of this are isolated collision domains and full duplex communication. The only source of communication jitter is the amount of time that an Ethernet frame is spent being buffered in the network interfaces of the nodes and of the switch. By limiting the bandwidth of the sending nodes this buffering delay can be controlled and it is possible to guarantee worst-case latencies. The scheduling analysis necessary for this approach is described in [Martinsson 02]. One of the nodes in the network acts as the network guard (NetGuard). This node is responsible for the scheduling of the real-time traffic. It also acts as a router for the non-RT traffic on the net. In the implementation of the switched scheduled Ethernet plug-in the scheduling functionality of the NetGuard is implemented by a CORBA object within the NetGuard node and all exchange of scheduling information between the nodes and the NetGuard uses standard CORBA/IIOP invocations. 152 5.18.2 Real-Time Channels The hard RT Ethernet approach primarily considers periodic hard real-time traffic. A periodic real-time communication between a sender node and a receiver node is denoted a real-time channel (RTC). The real-time channel is represented by a collection of properties describing the real-time traffic. It includes: • • • • • Sender node. The sender node can be identified by Ethernet address or IP address. Receiver node. Can be identified by Ethernet address or IP-address. Frame size. The amount of data to be exchanged. Frequency. The maximum frequency for the periodic communication. Maximum latency. The maximum allowed communication latency. If it is assumed that a global clock is available then more timing information needs to be recorded, e.g. offsets. The Real-Time Channel is able to relay the necessary information in order to allow the execution of a servant operation at the server side (e.g. Object key, operation, service context, etc.). Within CORBA the RTC maps to a connection. It is assumed that a private, preestablished connection is used for each RTC. 5.18.3 RT Ethernet Pluggable Transport The RT Ethernet approach is structured as in Figure 54. An additional RT layer has been added between the IP layer and the Ethernet interface. The RT layer implements the traffic control for the worst-case scheduling. HARD REAL-TIME CORBA REQUESTS Hard real-time requests assume that the connection for the RTC has already been established. Making an HRTC request generates a GIOP request that is forwarded to the pluggable transport. The client that issued the request continues execution immediately, e.g. asynchronous one-way invocations are used. In the pluggable transport on the client side, the GIOP is preprocessed and the values of the method attributes are encoded into a special RT-Ethernet frame. The header of the RT frame indicates that this is a scheduled RT frame. This frame is then transmitted to the server. At the server side the RT-layer receives the RT-frame, detects that it is a scheduled frame, and passes it to the pluggable RT Ethernet transport. In the transport 153 the values of the frame are extracted and the corresponding GIOP request is created and passed to the Real-Time ORB. For the sake of simplicity, it is assumed that clients are “well-behaved”, e.g. that they do not issue requests at a higher rate than what they have requested when the connection for the RTC was established. If this is not the case, one could require that the RT Layer should be responsible for the necessary traffic control (buffering) to ensure this. CORBA Objects + Clients RT-ORB IIOP TCP IP RT Ethernet Interface Figure 54: The RT-Ethernet approach ORDINARY CORBA REQUESTS Ordinary CORBA requests work essentially in the same way as in ordinary CORBA. The main difference is that the RT layer adds an additional header to the frames indicating that this is a non-RT packet, and that the RT layer buffers and, possibly fragments the IP packets, to ensure the schedulability of the real-time traffic. At the server side the RT layer buffers and composes together fragmented frames and then passes them up to the IIOP/TCP/IP layers in exactly the same way as in standard CORBA. For the ordinary CORBA traffic two-way requests are supported. 154 The solution has a lot in common with the “Pluggable data-interface” solution proposed for the pluggable TTP transport. The main differences are that the communication is event-triggered rather than time-triggered and that ordinary CORBA requests can be combined with hard real-time requests. 5.19 Platforms for CORBA Control Systems This section is a non-exhaustive list of hardware platforms where the hard real-time and minimum CORBA version of ICa can be run. The hardware platforms described can be used in systems with different demands from embedded soft real-time systems to fault-tolerant embedded hard real-time systems. 5.19.1 Embedded industrial PC Boards This type of boards usually have a small size factor (e.g. PC104 or HDD3.5” form factors) and provide interfaces for special equipment (e.g. GPS). Usually this type of boards is equipped with low power consumption processors and common interfaces to external equipment (serial, parallel and Ethernet). For obvious industrial reasons this type of boards usually comes with a compact flash memory storage device which serves the purpose of hard disk. Figure 55 shows two industrial PC boards with different form factors from Lanner Electronics. Figure 55: A PC104 and a HDD3.5'' form factor industrial PC boards (from Lanner Electronics) 5.19.2 Networked Device Servers Networked device servers are general purpose boards that use specialized hardware and processors. As industrial devices, this type of server is enclosed in sturdy casing and complies with immunity, emission and safety standards. The network device 155 servers use flash memory for the software system storage and are fully programmable. The usefulness of a device network server relies in the variety of their interfaces. One or more Ethernet / Fast-Ethernet ports and several serial RS232 and parallel and USB ports are common configurations for this type of devices. RS485/422 is also commonly found as they provide serial communication over longer distances to devices. Figure 56 shows a network device server board from AXIS Communications. Figure 56: AXIS Communications network device server (front and back sides) 5.19.3 Control Units Remote Control Units (RTU) are small computers used for automation of processes such us control of machinery or assembly product lines. They have either modular or integral input / output circuitry that monitors field sensors and controls output actuators according to the programmed control strategy of the unit. They are usually integrated into DCSs. Figure 57 shows the ELITEL-5 RTU from ELIOP where ICa ORB runs. 5.19.4 Time-Triggered Hardware TTP is a time-triggered network protocol based on a TDMA bus access scheme. There are several classes of TTP protocols ranging from TTP/A for low-cost systems to TTP/C for high speed network with high dependability requirements. TTP provides the predictability needed for hard real-time applications while keeping fault-tolerance capabilities. The TTP-Development Cluster hardware shown in Figure 58 (left) is based on TTPPowernodes mounted in a rack and with one TTPMonitoring Node for real-time TTP bus monitoring and download. Each TTP-Powernode is equipped with the TTP-C2 controller (AS8202). In addition to TTP, a broad variety of interfaces is supported: ISO 9141 (suitable for TTP/A, LIN, and ISO-K), CAN, digital I/O, and analog inputs. 156 The TTP X-by-wire box of Figure 58 (right) is a platform for rapid development of TTP-based distributed control systems in advanced automotive applications with highpower actuators. The box is an actuator control unit that offers hardware and software support for direct control of a brushless DC motor. Figure 57: A control unit from ELIOP Figure 58: A TTP development cluster (left) and a TTP X-bywire box (right) from TTTech 5.20 The ICa CORBA IED Among the interesting results of and platforms employed in the work of this Thesis is the CORBA IED (ICa PLC). The ICa PLC was developed during the IST-funded DOTS project. Further information about this project can be found in Chapter 7. The CORBA PLC uses the Power Substation Objects Subsystem on top of the ICa ORB embedded in the PLC of Figure 57. The Power Substation Objects Subsystem is the result of the mapping of the Abstract Communication Service Interface (ACSI) from IEC-61850 [IEC 01b] to CORBA. Such a mapping from an abstract interface to a real world mapping is denominated a Specific Communication Service Mapping (SCSM). 157 In other words, the Power Substation Objects Subsystem is a SCSM derived from the IEC standard ACSI. As will be explained in the next chapter, the development of these objects is an effort of domain engineering to develop a generic framework based on ICa for the development of power substation applications. The overall structure of the ICa PLC is shown in Figure 59. Figure 59: UML deployment diagram of the ICa IED A brief description of the purposes of the components shown above follows. • • • • • IEDApplication: it contains the main executable application packages. ORB: it represents the ICa middleware in its version tailored for embedded real-time systems. OperatingSystem: it contains the VxWorks operating system packages. AnalogPrimaryMeasure: it contains the executable application packages devoted to the acquisition and low-level processing of analog inputs. Configuration: it contains the configuration data of the application, in binary format 158 • • • IEDStartingCode: it contains the bootstrapping code of the operating system. IEDLoader: it contains the packages needed to upload and download the components located on FLASH1 and FLASH2. ObjectsData: it contains the persistent and temporal objects used by the executable components. Figure 60 shows a more detailed package diagram of the ICa-based IED for power substations. A description of the objectives of every one of the packages is presented in the next items. • • • • • SAS. It is a top-level package composed of all the packages that provide the SAS (Substation Automation System) functionality. It is composed of the subpackages in charge of the following functionality: digital input scanning, digital input supervision, digital input fast supervision, analog input supervision, digital output supervision and protection. Basic Services. It is a top-level package that comprises the packages that provide common services to the rest of the packages such as configuration management, subscription management, event management, real-time clock, utilities, autotest, hardware layer interface and start up procedure. Communications. Also a top-level package that includes the packages needed to provide communication services to the IED. The protocol stack, typically contained (in standard IEDs over telecontrol protocols) in the communications package, is provided by the Operating System via the ICa ORB package. On the other hand, the drivers for Ethernet communication are implemented as BSPs, as part of the hardware layer interface package. The communications service sub-package provides concrete implementations of the ACSI interface whereas the SCSMCORBA provides the mapping to the CORBA environment. ORB. The ORB package contains an instance of the ICa ORB for real-time embedded systems. HMI. Contains the packages devoted to the implementation of the local IED human-machine interface. 159 Figure 60: UML package diagram of the ICa IED 160 Chapter 6 A Methodology for CORBA Control Systems 161 6.1 Need of a Methodology The final objective of ICa is to let developers build the real-time distributed systems they envisage. But the mere existence of a framework like ICa does not ensure that a particular working system can be built. There is a need of a methodology that helps in the development of real-time CORBA distributed systems. Achieving the behavior regarding QoS, dependability or safety a mission critical system might need is not guaranteed by the simple use of ICa software. Sound engineering processes are necessary. Therefore, best-practice guidelines must be provided by ICa to build systems from an engineering point of view. Further, there is also a need for software reuse to be integrated in the bottom line of the system development process. Distributed real-time systems are highly complicated and very difficult to build. ICa systems must be constructed from a core of software assets that can be reused for a domain of applications. Software reuse reduces the cost, time and risk needed for development and system deployment. At the same time, it requires an investment for reuse as software must be specifically planned for that objective. Architectural frameworks like ICa only provide limited support for system design. Consequently, system design is an issue to be solved and, as ICa is an object-based framework, UML seems the standardized modeling language upon which an ICa systems engineering methodology must be built. One of the most interesting features of UML is that it is not a closed language. It makes provision for language extensions. In the design of CORBA hard real-time systems there are two UML extensions that are most useful; the CORBA UML profile [OMG 02m] and the UML profiles for realtime systems [OMG 04d], [OMG 05a]. These three profiles provide explicit support for CORBA based systems and for real-time systems. Developers of ICa applications are encouraged to make use of these facilities when planning their systems. UML is only a modeling language and is not in itself a methodology. In defining a methodology a software development process is needed. In some cases the modeling language and the process are tightly coupled but in the case of UML they are not. It is needed only that the development process is use case driven, architecture-centric, iterative and incremental. The Integrated Control Methodology (ICm) for CORBA Control Systems (CCS) is based upon the UML and a software development process that respects the constraints imposed by the use of the modeling language. The development process is based on domain engineering as the foundation for the systematic creation of domain models and software architectures. 163 6.2 Domain Engineering for CORBA Control Systems In it widely recognized that system software architecture affects organizations as much as organizations affect architectures [Bass 98] and that significant investment must be made until a satisfactory software architecture is obtained for any particular domain. In fact, that is the basic premise of the architecture business cycle (Figure 61), the relationship between the architecture and the organization. If this Thesis were focused to the resolution of a specific single control problem, significant effort would have been employed for just a single solution implementation. In engineering it is often needed to supply some degree of generality to the solutions achieved and precisely that is what ICm provides. It is necessary to address how generic heterogeneous complex distributed control systems are developed following software architecture practices in the domain of distributed control systems. Architect’s Influences Architect •Requirements and qualities )) •(end user and developing organization Architecture •Technical environment •Architect’s experience System Figure 61: System architecture business cycle To be able to generically develop for a range of systems, they must share a common reference model. In most DCSs there is an underlying common model that allows us to build systems by specifying the set of common software components to be used and the data flows between them. In this context, the reference model becomes a reference architecture. If a core set of software assets is developed to implement systems, the reference architecture becomes the software architecture for a type of systems. This means that DCSs product lines can be built as product families. Construction is made 164 by reusing the core software artefacts that represent the assets of a software product family 9 . Nowadays, there are different industrial areas where DCSs share reference models. This is something that occurs frequently in mature domains. In many cases, there is also agreement in the reference architecture for a domain. Unfortunately, in the DCS industry, there is not such an agreement regarding the implementation of the architecture and thus, thousands of different pieces of software are used for every system implementation. This has a direct relationship with the architecture business cycle. In some cases, there are system requirements that force the use of certain elements for technical reasons (e.g. a real-time operating system) while in others, the factors that affect the mapping of architectures to systems can simply be related to economic stakes (e.g. raising of commercial barriers for competitors). This affects system architecture and, as shown in the architecture business cycle, the architecture will affect the buying organisation which will be married to a developer / vendor and will suffer for increased maintenance costs among other drawbacks. The only way to avoid this is to think in architectures that are implemented in a standardised way. One of the benefits of standardisation is that it forces the people involved in the standard specification process to think about multiple systems, they have to meta-think in what systems need and to provide common interfaces for the domain of application. This is the basic idea of domain engineering. In this and the previous chapter, it has not been forgotten that there is work in progress in the OMG and that there are hundreds of people that may already be working in the specification of solutions to problems that exist in the distributed control systems domain. Thus, in order to build ICm an effort has been made to find which CORBA specifications and services can be employed in the control target domain. The final idea is to organize a collection of reusable software assets to be employed in CORBA Control Systems. A primary effort has been a traversal research across domain specifications. This is a typical problem of large standardisation or specification bodies that group large amounts of people. It is difficult to know what other groups are doing and in occasions it results that the same problem is solved independently by several groups. Although rework can be avoided in some cases just by learning what others do, it was found out that there was not an organised effort into the OMG to deal with the issues of distributed control systems. In order to develop core assets for control systems, the A product line is a collection of products for a market niche. A product family is a collection of products that share assets (designs, components, etc.). The ideal situation is the construction of product lines as product families as this maximizes the effectiveness of the engineering process (time, cost and quality). 9 165 ICa infrastructure was developed to build control systems upon it. In this case, the infrastructure, ICa ORB, is a real-time CORBA compliant ORB enhanced for DCSs. Although CORBA was not initially foreseen for industrial systems, it has specifications for real-time and fault-tolerance. Unfortunately, these specifications, specially the real-time specification, miss to treat some of the most important points for control systems. To overcome these difficulties, a completely new CORBA ORB was developed to incorporate functionality for control purposes. Specifically, as explained in the previous chapter, many areas of the control domain have been addressed: sources of overhead or non-determinism in the ORB, communication models, CORBA control loops, event-triggered systems versus time-triggered systems, the notion of time in control systems, etc. (see all details in Chapter 5). With these items considered, it is possible to have an infrastructure upon which core assets for control systems can be built. It is not possible to have assets with real-time or other control properties if the underlying system, the middleware, does not support them. In an attempt to formalize the process of designing for reuse and making it more repeatable, verifiable and methodical, the focus has been put on domain engineering [Harsu 02]. Domain engineering, when compared to application or single-system engineering, has, at minimum, these distinct aspects: 10 • Multiple System Scope. Control system engineering involves designing the best solution for a single intended application (e.g. a control system for a specific system to be controlled). CORBA Control Systems domain engineering involves designing assets that will optimise –or at least enhance– the overall engineering process for some set or class of distributed control applications. Unlike developing a single control system, where there is at least in principle some optimum (or at least adequate) single solution to be designed, in domain engineering there is an attempt to model the space of solution alternatives themselves. • Smaller than System Scope. Control systems engineering, presented with a single application context, implicitly has the job of designing the “whole solution” 10 . Domain engineering, while spanning multiple applications, has the “luxury” of picking what scope of functionality to address across those systems. For example, a domain engineering effort addressing the family of custom applications created within a telecommunications company could Obviously for the context of application. 166 focus specifically on the software implementing a specified class of features (e.g. call forwarding, call waiting) within those overall systems. This means the CORBA Control Systems domain engineer has several scoping decisions that are not part of the ordinary control systems development process: scoping the “range of coverage” for the components to be developed, both in terms of the type and number of control systems within which the components will be applicable (the “market” or reference model) and the specific features of those systems to be addressed by the components (the “coverage” or reference architecture). If these and other related methodological issues are not dealt with intentionally in the process followed, they can compromise what would otherwise be workable software engineering results. These basic concepts therefore turn out to have pervasive impact on all the phases, activities and work-products of the domain engineering life cycle. As with any work done with some perspectives of future, several overlapping and nested domains must be confronted. CORBA object-based reusability empowers developers with what we call progressive domain focalisation [Sanz 99a]. This means that development can be focused in a domain without sacrificing extensibility to broader domains and without hampering further concretion in narrower domains. 6.3 Methodology Overview The ICm methodology for CORBA control systems is based on domain analysis, design and implementation. Working on domains instead of applications or systems yields several advantages: • • • CORBA control systems analysis collects relevant information from existing systems of a control domain. The analysis identifies commonalities and differences and provides understanding of the relationships among the components of the domain. Most important, analysis bounds the domain. The domain model resulting from analysis is used to carry out CORBA control systems domain design. The main product of domain design is a generic architecture for CORBA control systems in a certain domain. The generic architecture arises from the domain knowledge gained in the analysis phase and from the study of existing classifications of common architectural styles and system patterns. CORBA control systems domain implementation consists in the elaboration of patterns and repositories of reusable components for the control domain. 167 Components of the domain have been identified from the analysis and design phases and from its main product, the architecture for the domain. In accordance with domain focalization, the proposed methodology allows to develop different types of solutions for CORBA control systems. They have been classified according to the different types of entities that can be constructed while developing CORBA Control systems. The solutions are classified as follows. • • • • ICa Objects. They are the basic components of a CORBA Control System solution. Objects can be thought of as software objects in the sense of Object Technology and can be distributed (CORBA objects). ICa Patterns: They are basic object organizations that provide some abstract functionality. They are instantiated in real-time systems (typically by means of ICa Assets). ICa Assets. They are software artifacts made of composed ICa Objects. While objects provide simple functionality, assets are able to offer complex services to systems or subsystems. Tipically are pattern reifications. Systems and Subsystems. Systems are composed of assets. A system is a CORBA Control solution that is made of from the integration of assets. In ICm composition is achieved by means of sticking together the listed solutions by use of the ICa platform middleware. The processes involved in the creation of CORBA Control Systems are depicted in Figure 62. Product Line Analysis System Systems Domain engineering Application engineering Commodity Assets Objects Principal ICa Figure 62: Integrated Control methodology process 168 6.4 CORBA Control Systems Domain Analysis Domain analysis in ICa includes domain scoping and identification, the systematic analysis of commonality and variability for a family of systems across the distributed control domain and the construction of the feature or requirement set for the family of systems. The objective of the analysis is to identify the elements that are common to all the members of the CORBA Control Systems family and to identify those elements that are variable across members of the family. The first step into domain analysis consists of the identification and scoping of the domain of application. 6.4.1 CORBA Control Systems Scope As a first step of the methodology analysis phase, the engineering domain must be identified and bounded. Usually, domain focalization must be used to dynamically change the scope among the different elements that form the system (subsystems, assets, objects). Shifting the scope gives the architect or engineer the chance of reducing the size of the domain and to concentrate on the task and technology that must be employed for a subsystem. Further, domain focalization helps to find solutions that are valid across domains. During this chapter, ICa and other systems will be used as examples of the use of the methodology to build a framework for the family of CORBA control systems. Thus, the ICa methodology will be used to explain the domain engineering behind ICa. ICa technology is a system class solution in the list of solutions that the methodology helps to achieve. However, this is a solution that has been achieved as a result of domain engineering and application engineering for a family of systems as ICa provides a framework for the development of solutions in the CORBA Control Systems domain. ICa is used as the core example of scoping the domain for a family of systems. The scope of ICa is that of distributed control systems. This is an enormous domain and is, perhaps too, simply defined. It is because one of the objectives of this work has been to demonstrate that a single technology can be employed to address many problems of distributed control. Thus, the ICa scope comprehends from the smallest embedded distributed control systems to large enterprise / industry wide control systems. ICa technology (broker, objects, assets, etc.) can be used for a wide range of distributed control systems because it has been designed to support a high number of points of variability that are used to implement specific control systems. Domain 169 engineering (see Section 5.2) has been used to identify points of variability in the distributed control systems domain. 6.4.2 ICa Domain Analysis The most difficult part when performing domain analysis is to identify the points of commonality and variability for a family of systems and to cluster them into a feature set. ICa provides a framework in which points of variability allow to derive specific / generic solutions to control problems by means of application / domain engineering based on the common vision provided by the ICa ORB. Commonalities are less difficult to determine as common components of systems can be in most cases easily identified by the examination of multiple systems used as exemplars. As shown in Figure 62 different types of solutions can be developed by the use of the ICa framework. This is because ICa has been thought as a generic framework for the whole range of distributed control systems. Thus, with ICa and its methodology it is possible to • • Perform domain engineering to develop a framework on top of ICa to solve the problems of a specific control domain. For example, domain engineering can be applied with ICa to develop a framework for time triggered control systems. This means knowledge in the time triggered systems domain is used to develop an infrastructure for a product family. Perform application engineering on top of ICa to develop specific control solutions. This can be achieved by hard coding the points of variability in ICa to solve a specific control problem. This is exactly the same as customizing the ICa framework to develop a distributed control application. Domain definition and scoping is one of the hardest parts of domain analysis. Domain analysis uses existing examples and the knowledge from domain experts and other sources of information to provide a domain model from which to instantiate application systems. However, in the case of distributed control systems this model is very complex as the domain is very wide. At the same time, the domain model does not describe how good control systems are built. This problem has been solved in this work by developing both the ICa ORB framework and the ICm methodology. The methodology helps capture domain knowledge and to plan software with / for reuse while the ICa framework is the means to instantiate the objects, assets and systems of the domain and gives to developers generic points of variation for distributed control systems design. 170 For this approach to work, the ICa broker itself has been developed by using domain engineering so it supports the features needed for the distributed control systems domain. Notice that we have specifically used the word “features” as opposed to use cases. ICa ORB provides features to objects, assets and systems built on top of it. These elements can be part of other domain engineering efforts (domain focalization has been used to narrow the focus of ICa) or can be developed to implement single systems by direct application engineering on ICa. The approach is presented in Figure 63. Control System A Generic Control Domain Engineering Use cases Control Framework A Use cases ICa features Use cases Specific Control Domain Engineering Control System B ApplicationEngineering Figure 63: Domain engineering and analysis with ICa In the figure, ICa features drive the methodology towards development of control frameworks and systems or applications. At the core, ICa features have been achieved by domain engineering of the distributed control domain. This domain engineering is labeled as generic as features are provided for a wide span of distributed control systems. From the core of assets that ICa provides, traditional domain analysis can be employed to focalize on other control domains, e.g. telecontrol of power substations. Likewise, by performing application engineering it is possible to directly implement systems with ICa. Chapter 7 provides examples of real systems that have used specific control domain engineering and application engineering on top of ICa. 171 The advantage of this method of analyzing a domain is that ICa has already performed a great deal of the domain analysis. When designing a distributed control system, a great deal of decisions must be made. These decisions pertain to different domains: the control systems domain (and its sub-domains, e.g. the automation systems domain or the embedded systems domain, etc.), the distributed systems domain (in the case of ICa, this is fundamentally the CORBA domain), the real-time systems domain and finally the domain whose problems the domain engineer needs to deal with. ICa and its methodology allow the engineer to focalize only in the domain he needs to address (for instance the control of power substations domain) and forget about the remainder domains as they are already contemplated by the ICa framework. The only work to be done regarding the collateral domains is to select the needed features from ICa ORB. To allow developers concentrate on a specific control domain, the ICa ORB domain analysis has decomposed the problem of building distributed control systems (see Chapter 5) trying to provide maximum satisfaction regarding the following requirements: • • • Control over processing, communication and storage resources to improve real-time features. Variable framework size to be fit into disparate target systems. Integration in heterogeneous environments. In the process of domain analysis and as ICa is an object oriented framework, it is recommended that UML use case notation be used in determining the commonalities and variability of the domain. Use cases, when expressed graphically, are not good at modeling variability because there is not a simple way to specify optional features (e.g. a control system is / is not supervised) or variant features (alternative ways to configure a mandatory or an optional feature). Use case modeling is recommended but always including its narrative form. The important thing is to correctly convey the information of the analysis. There are no silver bullets regarding notations for variability and frequently several techniques are needed depending on the final user of the information. [Trigaux 03] and [Bittner 03] provide a discussion of the basic techniques and their advantages and disadvantages. 6.4.3 Steps of CORBA Control Systems Domain Analysis with ICa As a summary of the steps involved in the domain analysis for CORBA Control Systems with ICm the following list is provided. A graphical representation of the 172 process is shown in Figure 64. Examples are given concerning the particular domain of Substation Automation Systems. 1. Identify the specific control domain. The distributed control systems domain is very wide. ICa has been designed to provide support for the whole range of applications in the domain. However, when developing for a family of control systems the domain shall be narrowed (domain focalization). An example of this process is found in the DOTS project (see Section 7.2.3) which develops a framework for Substation Automation Systems (SAS) on top of ICa. The designer only needs to focus on design for the SAS family of control systems while ICa provides a feature set common to SAS and other DCSs that can be reused with a limited effort. 2. Select analysis examples and identify the feature set for the control domain. To this respect, the commonalities and variability in the control domain (e.g. SAS) must be factored out. The common way of doing this is by examining a set of systems that adequately represent the domain. In the case of the DOTS project, the necessary knowledge was obtained from the experts’ knowledge that was put into the IEC-61850 standard for protection and control of electricity transmission and distribution systems and from domainspecific industrial project partners. 3. Cluster the features of the domain. Features must be clustered into layers, subsystems, assets or objects. This is often a hard part of analysis as heavy domain knowledge is needed. At this point domain entities, their behavior, interactions and functionality must be known. This process is carried out at a block level. Precise descriptions (e.g. interfaces) will be carried out in the domain design phase. It is possible that some of the identified entities are only analysis entities that will be transformed in design and implementation entities in later phases of the development process. 4. Identify the features that are supported by ICa ORB. Although with a great degree of flexibility, ICa guides the overall design of the reference architecture for a family of systems. This means that, at the end, the components of the family of systems are mapped to CORBA objects running on top of the ICa ORB. It is important to determine what features are directly supported by ICa. For instance, if a component needs to be multithreaded it will be necessary to use a threadpool and to associate it to the portable object adapter where the component runs. This implies that some design decisions are mapped directly to the ICa framework and, that the developer need not worry about them. More difficult-to-meet features are also supported by ICa. In the case of the DOTS project, requirements from IEC-61850 such as “The functions used today are well known but the standard shall be open for future 173 requirements” are met by ICa intrinsically. As a CORBA ORB extensibility, adaptability, scalability and other features are core concepts of ICa ORB. Focalize domain Identify feature set Cluster features Identify Ica features Steps of ICa Domain Analysis Figure 64: Phases of ICa domain analysis 6.4.4 Example of ICa Domain Analysis for SAS This section presents a summary of how ICa domain analysis has been applied to the domain of Substation Automation Systems (SAS) in the scope of the DOTS project. The analysis followed the steps designed in the methodology for this purpose. Domain identification. As it is easily understood several domains overlap. Pure domains are very rare to find as usually there is common functionality that is useful across domains 11 . This is what ICa takes advantage from. In the case of building a framework for SAS, the domains are the following: distributed systems domain, realtime systems domain, reliable systems domain, embedded systems domain and the one with real interest for end users, the SAS domain. ICa relieves the pain from developers for all domains except from SAS (with limited support for reliable systems). Then, it is possible to focalize on the problems of substation automation and control. Figure 65 shows an overview of the complexities of the SAS domain. The figure shows a collection of components (from IEDs to remote supervisory control) that can take advantage of ICa to provide integration in a standardized way. 11 This is in fact the reason why a transversal approach like ICa can have any success. 174 Operation terminal / Configuration terminal Remote RemoteSCADA SCADA Front end High speed links ,network Low speed links IED RTU Maintenance terminal Physical devices Controlled Plant Figure 65: General view of the SAS application domain. Feature set identification for the SAS domain. For this purpose it is necessary to take into account exemplar systems, domain experts knowledge, common systems requirements and all the information available to derive a feature set for the applications in the domain. In the case of SAS and for the DOTS project, reliance was put in the feature set identified by the IEC-61850 experts committee. Features in this IEC standard are grouped under the concept of function. A function is a task carried out by the SAS which exchanges information with other functions to carry out its work. With this information it was possible to develop a set of features related to the SAS functionality: control functions, metering functions, disturbance recording, protection functions, switchgear functions, etc. Figure 66 shows how SAS functionality is structured at the different levels of the substation. SAS feature clustering. Fortunately, in the case of the DOTS project, SAS features were already classified (at a high block level as needed in domain analysis) by the IEC-61850 standard. This clustering together with a grouping of feature classes is also shown in Figure 66. The figure clusters the features according with the different levels (layers) identified in a substation (e.g. the process, bay and station levels). Notice that in the case of distributed control systems feature clustering is usually influenced by the physical devices of the system and by the layered architecture of distributed control systems. A finer clustering of features is achieved by examining the 175 functionality of each of the devices. A higher level of feature clustering can be achieved by examining the way devices collaborate to perform systems functions. This last case is what is considered for the domain of SAS. Features identification. For the given analysis, the features to be provided must be sorted out. These are the features that may be found in most distributed control systems. In the case of SAS, these features were related with the capability of ICa to be integrated with other (legacy systems or not) systems, its capabilities to be embedded in devices, and with their abilities to control concurrency and communication transport aspects of the system. The completion of the four steps explained before is what makes up domain analysis for SAS. After this, it is possible to employ the information of the analysis to devise an architecture for the domain. That part of the methodology corresponds to the CORBA Control Systems Design of the ICa methodology. Figure 66: General structure of a substation automation system as shown in the IEC 61850 Standard [IEC 01c]. Level 0 = process, level 1 = bay, level 2 = station. 176 6.5 CORBA Control Systems Domain Design The objective of domain design is to take the results of the ICa domain analysis and to elaborate them to produce a generic design and implementation for a family of related systems. To produce such a design it is needed that the abstraction level employed during the analysis stage is kept also at this stage (in the sense that design is not being made for a single system). Design must result in an ICa-based software architecture that is reusable across systems. As happens with all software architectures, the resulting domain model of the design process must supply two things: • • A detailed partitioning strategy that defines the components of the architecture and the allocation of the domain feature set to them. A coordination and control model that defines the behavioral model of the software elements that compose the architecture. The former steps must be performed taking into account that ICa is based on an ORB. 6.5.1 ICa Domain Design and Implementation As said, while doing analysis, it is important to remember that ICa ORB is a CORBA ORB so architectures resulting from domain design will be influenced by the reference architecture of the OMA. However, the trick here is to remember that ICa is a framework for frameworks. This means that it is possible to build other generic architectures for distributed control systems on top of ICa (see Figure 63). In the ICa domain analysis, a partitioning strategy was already carried out. It was called feature clustering. That partitioning strategy was done in a coarse-grained style. Usually, in the type of systems which are developed with ICa, that partitioning corresponds to physical entities of the system. For instance, when analysis was made in the scope of the DOTS project, feature clustering was focused around the real-world entities that appeared in the system. Remote terminal units, operator terminals, switch gear, etc. were some of those entities. However, when going deep to design, feature clustering is too high-level. Then a more detailed partitioning strategy that deals with processes, interfaces, objects, connectors, control flow, etc. is needed. The second aspect of ICa domain design is the coordination model. It is not possible to design a good coordination model if the domain is not properly partitioned. Development with ICa means developing with distributed objects. If the entities of the 177 domain (objects) either are not well chosen or are too poor there is significant chance that the coordination model will not have the sufficient flexibility to fit a family of systems. On the other hand, if the domain object model is too rich, there is also a chance that the coordination model will be too complex. In this case, if the system is heavily distributed, this will result in a loose of performance due to excessive use of communications. Additionally, devices with scarce resources will be overloaded as they will have to instantiate more objects than needed. A good rule of thumb is to remember that sending a message to an (distributed) object is very expensive in terms of communications and computing at both sides. The exact richness that must be conferred to the domain model is a matter of experience. Too poor the domain model and the benefits of object orientation could not be reached. Too complex and the responsiveness of particular implementations will be a problem. The last point in ICa domain design is that of designing for reuse. Pure domains do not or rarely exist and most systems exhibit features that can be found in several domains at the same time. This is why ICa is useful; it takes advantage of features that are useful in different domains (e.g. the sub-domains of distributed control systems). Designing for reuse means planning in advance for reuse. The best recommendation that can be done for reuse is to keep an eye for elements of the architecture which are dependent of each other. Dependencies preclude reuse as it is a sign that the elements are tightly coupled. Next, the steps of ICa domain design are presented. 6.5.2 Steps of CORBA Control Systems Domain Design and Implementation with ICa The main results of ICa domain analysis were the identification of the domain, feature identification and the clustering of features (usually around devices) and the search of features that could be implemented by ICa. Here, the steps of converting those results into a generic architecture and implementation for a family of systems are introduced. 1. Subsystems identification. When developing with ICa, systems are made up of smaller sub-systems with defined missions. The first step consists of subsystem identification. This step is achieved by assigning clusters of features to subsystems and defining the different structures that make up the system. These structures can be conceptual, process, physical, control-flow structures or any other considered important from the software architecture 178 point of view to identify each subsystem. Whether the system is made up of subsystems or just ICa assets is matter of system size. 2. Asset identification. Assets are the primary means of reuse and subsystem construction. While at the subsystem level it is difficult to define interfaces that can be employed many times over, at the assets level there is more independence of the external constraints of the domain. So in the long run, it is better to design for reuse at this level (patterns play an important role here). An asset is a set of objects that collaborate to perform a system or subsystem function. Collaborating assets build up subsystems. 3. Object identification. Objects are the basic elements of ICa development. Objects collaborate to form assets. Thus, objects have only basic behavior. In creating objects, common object techniques can be used. Object attributes and persistency must be clarified as well as their operations and the degree in which the operations may vary so assets exhibit flexibility and can be used in different domains. Notice, that when moving from analysis to design, analysis objects can be decomposed into several collaborating design objects. 4. Interface types identification. So far, the system is defined by sheer definition of its constituent parts from different points of view. However, the interfaces of subsystems, assets and objects may be realized in two different forms: remote interfaces and local interfaces. Remote interfaces correspond to ICa objects while local interfaces cannot be directly accessed by ICa. The point here is that distribution is always expensive and should not be employed unless necessary. Additionally, making all system elements interfaces remotely accessible increases system overhead in performance and size so remote interfaces as small as possible must be defined. In general, objects do not have remote interfaces unless the system is small, the object is independent of other objects and the object performs functions which must be accessed from other distributed objects or assets. Assets tipically have one remote interface for the whole set of objects that form the asset. Similarly, sub-systems have one interface that allows access to subsystem functionality. Notice that lower levels may not have a remote interface if they are part of upper level elements (e.g. an asset may not have a remote interface if it is part of a subsystem and all the assets that make the subsystem run in the same process). A final note is that of programming style. When dealing with local interfaces, fine-grained interfaces can be developed (e.g. including get or set operations / methods for object attributes) as the overhead of sending a message between objects inside a process can be dismissed. However, remote interfaces must exhibit a coarse-gained programming model. Each operation of an interface must contain all the arguments needed to perform complex functionality. This will minimize network usage although the programming model is more involved. 179 5. Create behavior 12 . The pre-final step consists in designing the allocation of the features identified in domain analysis to subsystems, assets and objects. For this, the distinction between local and remote is vital, the fewer remote components in the system the better. Behavior is also related to implementation as determining the capabilities of the domain framework in a specific application is a matter, for instance, of the device resources where the application runs. Thus, in many cases a reasonable option is to devise the framework in such a way that ICa behavior can be parameterized for later application engineering. 6. CORBA Control Objects framework implementation. At this moment, a distributed object framework for a family of distributed control systems pertaining to a domain has been devised. It is time now, to proceed with implementation. Notice that analysis and design of the domain objects has already been done and that this step is devoted to the implementation and tests of framework objects of the domain (e.g. those objects of a domain layer) and not those corresponding to a specific application. The problem is that the framework implementation must be used in a variety of devices with different amount of resources. In this case, some domain engineering for control devices inside of the main process of domain engineering for control systems is needed. A device abstraction layer is needed so a common interface to all the devices of the domain is designed. Such an interface isolates the domain objects from specific hardware and operating system details. Notice that in the previous steps all the design is related to object oriented design. Figure 67 shows a graphical representation of the steps associated to design and framework implementation. Objects are not standalone parts of a system. Objects collaborate and thus the coordination model of the system is devised along the partition model of the system. In other words, a system cannot be split into objects without thinking how those objects (be them the subsystem, assets or ICa objects) coordinate. 6.5.3 Example of ICa Domain Design for SAS In continuation of the example relative to domain engineering for Substation Automation Systems, the steps representing the ICa domain design for a framework for SAS is presented next. In practice, steps 4 and 5 can be overlapped in time as system behavior can be used to obtain interface types. 12 180 Identify assets Identify subsystems Identify objects Identify interfaces DocumentList FileMgr UI Repository Document add( ) delete( ) fetchDoc( ) sortByName( ) DocumentList name : int docid : int numField : int get( ) open( ) close( ) read( ) sortFileList( ) create( ) fillDocument( ) MFC FileList fList FileManager add( ) delete( ) read() fill the code.. 1 DocumentApp Document rep Repository RogueWave GraphicFile (from Persistence) File Persistence File read( ) GrpFile name : char * = 0 FileList readDoc( ) readFile( ) read( ) open( ) create( ) fillFile( ) global Subsystems Objects Interfaces Implement framework Create ICa behavior Framework implementation ICa behavior Assets Steps of ICa Design and Implementation Figure 67: Phases of ICa domain design and implementation Subsystems identification. When the example of domain analysis for SAS was introduced, a coarse-grained partition of the SAS system was made. Basically, the feature set for the domain was identified and then, the features were clustered into layers of similar functionality. For SAS three layers or levels in SAS terminology were identified. These were the process, bay and station levels. The subsystem identification digs deeper into these levels to identify the subsystems that compose each of these levels. As an example of subsystem identification, focus will be put on the bay level or Level 1. Level 1 corresponds to the bay units, the PLCs or IEDs placed on top of the process level. Here, for the subsystem identification, the point of view has shifted slightly from the mere functional features of the system, as considered during the analysis, towards a composed view that includes also implementation aspects of the layer to be built. Figure 68 shows how implementation (at the design level) and functional features of the system are related. 181 Figure 68: SAS Level 1 subsystem identification For the subsystem identification of the layer it was important to decouple SAS functionality from the communications technology to be used. This is the same as separating the ICa ORB technology from the SAS functions of the PLC. The upper figure shows the SAS, Communications and ORB packages (or subsystems if an UML stereotype is used) designed with this purpose. The HMI subsystem controls the specific human machine interface of a particular PLC or IED. The separation between the SAS, Communications and ORB subsystem is important as a framework for SAS based upon ICa technology is being built and the advantages of the real-time CORBA technology must be preserved. The separation of subsystems helps to modify SAS functions without bothering about the Communications and ORB subsystems (as long as the change does not affect distributed objects). Likewise, the Communications and ORB packages could be changed without the SAS functionality being affected. For 182 instance, the framework does not depend of any given ORB vendor. Also important is the separation between the SAS subsystem and the Basic Services subsystem. The latter subsystem is close to hardware and to the operating system and isolates the SAS functionality so it can be reused in different devices with different capabilities. In the case of the ORB subsystem, the ORB package is the real-time version of ICa ORB. The choice of the real-time embedded version of the ORB is imperative because a bay level unit in a SAS application is involved in critical distributed operations of the substation (e.g. interlocking) in an environment with limited resources. Assets identification. Assets identification is also shown in Figure 68. Inside each subsystem, the assets have been denoted as smaller UML packages embedded into subsystems. When analysis was considered, it was said that assets were the basis for reuse. In this case, assets are related to SAS functions and thus, a specific IED or PLC can be tailored to support or to lack some SAS functions. This way a product line for SAS with products differentiated by features can be created. Notice that the Communications package depends of the SAS package. Basically, the contents of the Communications package change by recompilation of the ICa IDL files needed for a device of the product line. The stub code is automatically generated by the ICa IDL compiler. Object identification. Object identification is a matter of focusing into the assets and finding the entities that collaborate for each SAS function. This is much an effort of object oriented design. Design classes from OOD must be found in this step. Notice that domain design is being dealt with in this section but, at the same time, object oriented analysis and design must be done here. Moving from object analysis classes to design classes creates new entities in the system. For instance, how dependent subsystems communicate must be elucidated. The use of the façade pattern for subsystem interfaces is a good means for solving this type of dependencies. Interface types identification. Once the design classes (or runtime objects) of the assets, subsystems, etc. have been identified, it is time to work out their interfaces. This includes understanding the dynamic behavior of the objects in the domain and deducing from it the exchange of object messages. In the case UML is used, the sequence of messages that conform the interface of an object can be deduced from interaction (sequence of collaboration diagrams) or from activity or state machine diagrams. Create behavior. At this point, decisions regarding what is directly related to the distributed system and what is not must be taken. In the case of SAS, a very specific subsystem, Communications, was devoted to ICa ORB related entities. Inside this package the elements mapped for the IEC-61850 to CORBA have been identified. 183 These elements are the ones which either are accessed by the SAS subsystem functions or that access the SAS functions under remote client invocations. In other words, the Communications Service asset of the communications subsystem implements the abstract entities of the IEC-61850 standard that describe the architecture of an Intelligent Electronic Device for SAS. As shown in Figure 69, an IED is an aggregation of logical nodes which can be of different types. Figure 69: The ACSI implementation from IEC-61850 As can be easily understood, different device models will support different functions for SAS. Even the same device models support different SAS functions in the same substation. For this reason, the devices are configured by means of an XML file and a configuration tool. This tool specifies the number and type of the logical nodes of the physical hardware device. There are many different types of logical nodes that collaborate to perform SAS functions. Examples of these logical nodes are circuit breaker controllers, switchgear interlocking managers, station-level alarm and event 184 managers, etc. Depending of the functions assigned to the device, different ICa objects will be created at start-up when the configuration file for the device is loaded. At the same time, ICa solves several problems of the real-time properties of the SAS domain. These are the properties related to storage, processing and communication functions. This functionality also depends on the features available in the devices where ICa ORB and the SAS framework are embedded so they are easily configurable. These features are what were previously called variability points of ICa ORB. Behavior is created with ICa by deciding what ICa ORB communication resources can be used by SAS applications, what level of concurrency is needed for the incoming requests to a device and by deciding the available storage space for incoming requests. It is at the implementation level, with the concrete devices and resources in them, when the configuration of the variability points of ICa ORB take greatest importance, although nothing precludes determining these parameters at this stage for all device models. SAS Control Objects Framework implementation. The last phase of SAS domain design is that of the framework implementation. In the case of SAS, framework implementation is mainly related to the construction of the different logical nodes that make up SAS systems according to the IEC-61850. Among the problems of implementation of the object framework for the domain are those of making a portable development for all the devices in which the objects must run (e.g. different models of PLCs) and those related to implementing the different configurations which have been devised for the framework (e.g. optional features). 6.6 CORBA Control Systems Implementation At the end, the objective of domain engineering is to better construct the systems of a certain domain. If done correctly, domain engineering has obtained a set of assets and a framework that can be reused many times over with better guarantees for the systems’ quality attributes compared to those obtained by building systems from scratch. With the use of ICa domain analysis, design and implementation the core of subsystems, assets and objects for a domain can be transformed into specific running distributed control systems by means of application engineering. At last, the implementation of a CORBA control system with the ICa methodology for a domain is largely a matter of selecting the required features from the domain framework and 185 of configuring its variability points. The objective of the methodology is producing as few ad-hoc implementation as possible. 6.6.1 Steps of CORBA Control Systems Implementation with ICa CORBA Control Systems implementation with ICa is related to application engineering but in this case, after ICa domain engineering, application implementation is reduced to a minimum. The following items present the steps that direct application implementation with the ICa methodology. 1. Identify the requirements of the system. The first phase or step to carry out consists in finding out the requirements of the system to be developed. Finding the requirements is important because it helps to define the functionality of the system. Notice that if, for example, requirements are found which are not supported by the framework developed during domain engineering it might be time to perform more domain engineering in order to add new features to the domain framework. In practice, this means that the domain framework is built in an iterative process while more and more systems are built and more knowledge is obtained. 2. Match system requirements to domain framework features. When a framework for a domain of control systems was built, there were features already supported by ICa. In the same manner, when building a specific control system there will be many requirements which will have already been addressed by the subsystems, assets and objects of the control framework and by the broker itself. 3. Identify missing features. The framework developed during the domain engineering effort must try to thoroughly address the problems of the domain. However, there are situations in which this is not possible. For instance, there might be a requirement to use tools from a specific vendor at the advanced control network level for political reasons or, modestly, the object framework engineer did not envisage some features. In this case, the missing or additional features must be identified in order to implement them in the final system. 4. Perform application engineering. Finally, it is the time of the application engineering effort. Application engineering is divided in two efforts. The first one is basically an effort of deciding what subsystem, assets and objects from the domain framework to include in the control system being built and of configuring them. The second one consists in the development of the 186 additional functionality not built into the object framework for the domain. For this latter effort, a traditional development process can be followed. Figure 70 shows a schema of the steps involved in constructing CORBA Control Systems implementations based on the ICa object request broker and on an object framework specific for a control distributed domain. Identify system requirements Match requirements and features Identify missing features Application engineering DocumentList FileMgr Document add( ) delete( ) fetchDoc( ) sortByName( ) name : int docid : int numField : int get( ) open( ) close( ) read( ) sortFileList( ) create( ) fillDocument( ) FileList fList add( ) delete( ) read() fill the code.. 1 rep File Repository (from Persistence) read( ) GrpFile name : char * = 0 read( ) open( ) create( ) fillFile( ) readDoc( ) readFile( ) UI MFC DocumentApp RogueWave Persistence global Application engineering and framework configuration Steps of CORBA Control Systems Implementation Figure 70: Phases of CORBA Control Systems implementation 6.6.2 Example of ICa Control Implementation for a SAS Application In the examples of the previous sections it was explained how a framework of control objects for the domain of SAS applications was built. In this section, an example of the benefits of making application engineering for a particular SAS system with the help of that framework and of ICa ORB is presented. The section contains the steps of ICm implementation particularized for a SAS application. Identify the requirements of the application. The SAS distributed control application was built to control one high-voltage line of the Red Eléctrica Española 400 kV open-air park substation in Villaviciosa de Odón near Madrid. Figure 71 shows the electrical 1-wire diagram of the substation with the line to control inside a green marquee. The line is composed by three circuit breakers, two line switches, four position switches, two bar switches, four measuring current transformers, two line 187 voltage measuring capacitive transformers, two blocking coils and an associated lane control room. Among the requirements of the SAS application are the following: • A group of requirements supported by deeply embedded devices at the bay level: • Physical device self-checking. • Control functions. • Event management. • Protection functions. • A group of requirements to be supported by substation level devices (including substation computers and the operator interface tool): • Still-alive image acquisition. • Network management. • Time synchronisation. • Software and configuration management. • Operative control mode of Logical Nodes. • Setting. • Alarm management. • Station-wide interlocking. • Automatic switching sequence. • Management, storage and display of the still-alive images. • HMI to display the substation window, the bay window and the maintenance window. • A group of requirements supported by the System Configuration tool, working offline: • Generation of configuration files. • Load and storage of SCL files, IEDs capabilities, layout of a power substation and layout of substation primary equipment. • Graphical display of substations and bays layout with possibility to configure the line diagrams. • Activation / deactivation / configuration of Logical Nodes and functionality. • The application has real-time requirements for all control related devices except for the operator interface tool, which is developed in Java language (the SAS framework is developed in C++) and uses the Borland’s Visibroker commercial ORB for communications. 188 In summary, the SAS application for the lane contains two IEDs placed in the lane control room which assume control and protection functions. There is also a substation computer placed at the substation control room which runs the HMI and is capable of undertaking some control functions. There is also a configuration terminal for the SAS application on a portable computer and all the systems are hooked up by a fiber optics network. Access to the lane control room is controlled by the HMI with direct digital live-video feed. Figure 71: The REE substation in Villaviciosa de Odón Match system requirements to domain framework features. The second step consists of matching the requirements of the SAS system for the REE substation against those supported by the framework developed on top of ICa for SAS systems. As the developed framework is built on the concepts of the IEC-61850 standard, it results that the matching is performed by identifying the logical nodes of the framework that support the requirements. The SAS framework addresses all the control and protection requirements for the SAS application by the use of logical 189 nodes such as circuit breakers, interlocking managers, etc. What this means is that the control part of the application is solved by configuration of the framework for the lane to be controlled. Regarding real-time features of the distributed control system for SAS, some requirements are directly solved by ICa ORB. The system can automatically be configured from configuration files about a number of features controlled by ICa ORB, (e.g. the number and priority of threads running in the IEDs or the use of a multiplexed connection for an IED). Also the system in the IEDs was configured according a client propagated priority model and the use of invocation timeouts was used to control error conditions. Further, ICa solves the problem of integration with external systems by means of ORB interoperability. The HMI is a Java application that uses an ORB from a different vendor. There is a direct integration between the two ORBs by the use of the common interoperability protocol IIOP. Further, the Java ORB could be cropped as remote method invocations in Java use IIOP as the underlying protocol. The use of Visibroker was decided to ease the work at the HMI side. Identify missing features. It is very difficult to directly implement a complex distributed control system just by the simple configuration of a framework. When moving from the control domain concepts to the real-world applications there are many situations that will have been overlooked. Some of them will be worth of refining the built framework as will appear in more systems so they must be incorporated to the domain model while others simply are specific to the system under work. In the case of the REE substation, there were the specific requirements of supporting access to the lane control room by a direct video feed and those of developing the HMI and system configuration tool interfaces. These are the missing requirements for the SAS application. Of course, the configuration tool and the HMI for SAS are worth an effort of domain engineering as can be reused in many systems but, in this case, were not considered part of the developed SAS framework based on the IEC-61850 standard. Perform application engineering. For the SAS system, application engineering is performed by building the appropriate configuration files for the substation devices. The framework objects that control the substation gear are built on system start-up from the configuration files. The additional work is that related to the development of the off-line configuration tool and that of the human-machine interface. It is important to notice that even if the framework does not support the HMI it greatly reduces its development cycle as the SAS framework on top of ICa ORB already gives the HMI a 190 domain layer (that of the SAS framework objects) that is employed together with the Java ORB in the development of the HMI. 6.7 Pattern and Architecture Exploitation ICa and its methodology provide a software framework into which CORBA Control Systems can be organized and developed. In the development of ICa, software engineering and architecture principles that eased the work of implementing the complex pieces of ICa software have been followed. Thus, the ICa ORB is composed of many subsystems that interface to each other and intensive use of Pattern Oriented Software Architecture (POSA) [Bus 96] solutions has been employed. The purpose of all these efforts has always been to base development on sound techniques so the final quality of the software will be good. However, even if performance is among the initial requirements for ICa ORB, there is always the perception that too much structure into systems results in a loss of performance compared to what we might call “dirty code hacks”. First, it should be noticed that, in our experience, there is no way to produce, maintain and evolve a large production system which is built around unstructured code and blurrily partitioned subsystems. Secondly, there is the fact that it is not possible to develop long-life systems without being involved in an effort of domain engineering. A system that must survive many years has the requirement to evolve and evolution is tremendously facilitated by domain engineering. The problem is that of modeling the objects of a domain but, once this is achieved, the benefits are enormous. Once domain engineering has been performed for a family of systems and the objects of the domain model have been identified it is possible to query about optimizations or performance. The idea to follow is something like “first structure then performance”. Nevertheless, it is also good to recognize that unpleasantly optimized code or dirty hacks are sometimes necessary. The best way to do this is to put the correct objects in the domain model so optimizations are enclosed inside objects and do not affect the rest of the system from the development point of view. For instance, ICa ORB incorporates the concept of thread in its model. Initially the hopes of portable development were concentrated in the POSIX standard which should have avoided many problems of implementation. In practice, reality is that not every operating system implements the POSIX standard, it is partially implemented or simply it happens that the native services of the operating system perform better than their POSIX equivalents. As an example, this happens with the VxWorks operating system where threads are just tasks and where synchronization object acquisition performs 191 better when native operating system calls are used. ICa ORB has a thread object model which is optimized for each operating system. This effort of domain engineering results in a sustainable and evolvable improvement of performance. The idea is to conceptually firewall either the unstructured parts of ICa (because they are too close to a particular operating system implementation) or the modeled domain from other subsystems. As always, the pragmatic approach adopted in this work is shown with an example. Following the development of the SAS control framework and its exemplar system for the Villaviciosa de Odón REE substation some performance experiments were carried out during the works of the DOTS project (see Section 7.1.2). The experiment physical layout in the REE substation is shown in Figure 72. OIT IHMI0 3' 5' 1 6 IED2 IED1 XCBR20 XSWI25 2 5 3 CILO0 4 Figure 72: Physical layout for interlocking tests The figure shows a simplified schema of two intelligent electronic devices placed in a lane control room of the substation (IED1 and IED2). IED1 contains, among others, a logical node (CILO0) which is devoted to the interlocking control of subsystem elements. In a substation some elements must be in certain controlled states before other operations on some elements can be performed. For instance, a circuit breaker cannot be closed if its corresponding switch is open. The process of operating the substation organs and of checking their state is called interlocking. IED2 contains a logical node representing a switch (XSWI25) and a logical node representing a circuit breaker (XCBR20). The box in the upper right part of the figure labeled OIT is the Operator Interface Tool running in the substation computer of the substation control room and implements a logical node (IHMI0) designed for user interface purposes. 192 The OIT runs the commercial ORB Visibroker whereas the IEDs with more stringent real-time requirements run ICa ORB. The orange box at the upper left corner of the figure shows two substation elements that must be operated in an interlocked way. The one on the left is a switch and the one on the right is a circuit breaker. For these experiments, the IEDs were configured with a supervision scan rate of analog and digital signals of 100 ms. The numbered yellow circles represent the sequence of actions to perform an operation. First, the operator commanding the HMI tool issues a command to the appropriate circuit breaker in IED2. The logical node representing the circuit breaker queries its associated interlocking control node (CILO0) on IED1. The CILO0 logical node queries then the associated switch about its state. This information is used then to inform the circuit breaker on the actions to perform. If the operation is rejected the user interface is updated (Figure 73). If the operation is allowed the circuit breaker logical node (XCBR20) closes the organ and the operator interface tool is updated (Figure 74). Case 1: operation inhibited º OIT. IHMI0 IED2. XCBR20 t1 t2 t3 t4 t5 t6 → XCBR.set_ctlVal IED1. CILO0 CILO.get_stVal IED2. XSWI25 → XSWI.get_stVal → opened ← ← ← Ethernet Frames - GIOP (time in microseconds) us 1 2 3 20.335.662 45.553.513 52.144.590 t1 20.406.110 45.614.671 52.203.864 t2 20.473.052 45.682.335 52.271.569 t3 20.610.142 45.832.117 52.416.208 t4 20.635.061 45.864.838 52.443.386 t5 20.659.827 45.888.113 52.467.953 t6 70.448 61.158 59.274 t1-t2 66.942 67.664 67.705 t2-t3 137.090 149.782 144.639 t3-t4 24.919 32.721 27.178 t4-t5 24.766 23.275 24.567 t5-t6 324.165 334.600 323.363 t1-t6 disabled rejected 4 58.703.645 58.768.336 58.832.201 58.968.215 58.990.098 59.012.211 64.691 63.865 136.014 21.883 22.113 308.566 5 64.313.420 64.374.433 64.442.697 64.582.785 64.604.270 64.626.445 61.013 68.264 140.088 21.485 22.175 313.025 6 74.245.842 74.314.190 74.383.529 74.523.270 74.544.635 74.566.733 68.348 69.339 139.741 21.365 22.098 320.891 7 79.532.090 79.594.563 79.666.425 79.804.676 79.826.270 79.852.838 62.473 71.862 138.251 21.594 26.568 320.748 8 95.386.618 95.449.168 95.518.615 95.657.546 95.679.531 95.701.697 62.550 69.447 138.931 21.985 22.166 315.079 9 97.130.014 97.190.656 97.262.690 97.402.515 97.424.028 97.446.218 60.642 72.034 139.825 21.513 22.190 316.204 10 avg. max. 106.980.377 107.041.147 107.110.436 107.250.338 107.272.837 107.295.188 60.770 63.137 70.448 69.289 68.641 72.034 139.902 140.426 149.782 22.499 23.714 32.721 22.351 23.227 26.568 314.811 319.145 334.600 Figure 73: Interlocking operation rejected The figure above shows the case when the closing operation of the circuit breaker is rejected because the switch is still in the open state. Time measures between during the flow of requests among the logical nodes are denoted t1 to t6. Average and worst 193 case times are presented in the right part of the table. Notice that the biggest lapse of time occurs in the t3-t4 interval. This is because the switch logical node has to physically connect by the wiring to the switch element in the substation and get its state. Afterwards, the switch logical node returns the switch state to the interlocking logical node and from there to the circuit breaker and HMI. Figure 74 shows the case in which the operation is not inhibited and thus the circuit breaker can be closed. Case 2: operation allowed OIT. IHMI0 IED2. XCBR20 t1 t2 t3 t4 t5 XCBR.set_ctlVal → t6 ← CILO.get_stVal IED2. XSWI25 → XSWI.get_stVal → closed ← ← close Ethernet Frames - GIOP us 1 2.499.720 t1 2.563.527 t2 2.631.717 t3 2.767.646 t4 2.789.230 t5 2.892.260 t6 63.807 t1-t2 68.190 t2-t3 135.929 t3-t4 21.584 t4-t5 103.030 t5-t6 392.540 t1-t6 IED1. CILO0 (time in microseconds) 2 3 5.365.568 7.835.573 5.429.650 7.900.292 5.493.665 7.966.715 5.630.963 8.104.121 5.652.693 8.128.383 5.762.258 8.232.135 64.082 64.719 64.015 66.423 137.298 137.406 21.730 24.262 109.565 103.752 396.690 396.562 enabled IED2. SD1 → accepted 4 9.296.164 9.361.910 9.427.648 9.570.560 9.596.547 9.698.891 65.746 65.738 142.912 25.987 102.344 402.727 5 15.425.039 15.487.889 15.565.181 15.704.305 15.726.077 15.829.857 62.850 77.292 139.124 21.772 103.780 404.818 6 19.646.169 19.709.996 19.784.617 19.924.838 19.956.273 20.064.049 63.827 74.621 140.221 31.435 107.776 417.880 7 21.981.022 22.046.448 22.112.679 22.255.135 22.276.619 22.386.585 65.426 66.231 142.456 21.484 109.966 405.563 8 24.083.936 24.149.677 24.218.979 24.362.593 24.387.501 24.492.481 65.741 69.302 143.614 24.908 104.980 408.545 9 28.529.510 28.593.594 28.661.929 28.803.099 28.824.590 28.930.021 64.084 68.335 141.170 21.491 105.431 400.511 10 avg. max. 32.992.953 33.060.292 33.125.986 33.269.912 33.295.873 33.405.228 67.339 64.762 67.339 65.694 77.292 68.584 143.926 140.406 143.926 25.961 31.435 24.061 109.355 105.998 109.966 412.275 403.811 417.880 Figure 74: Interlocking operation permitted In this case, the operation is permitted and the circuit breaker is closed. Notice that the total time to perform the operation is a little larger than in the previous case as the circuit breaker has to be physically closed. This can be seen in the t5-t6 interval which is now larger than in the previous case. About the validity of the result and the performance of ICa and the developed SAS framework it must be said that the requirement for operator operations was of about 1 s. This means that the results obtained are very good. Further, when compared to other substation control systems the figures found for this type of operations are similar to the ones achieved in the experiments. This must be considered, taking into account 194 that the SAS implementation is a first implementation and has not been optimized as other commercial substation control systems. These results validate the domain engineering approach taken for CORBA Control Systems with ICa, its broker and methodology. 195 Chapter 7 Exemplar CORBA Control Systems 197 7.1 Summary List of Funding for this Work The work in this Thesis has been developed under the umbrella of several international projects funded by the European Commission during the ESPRIT and IST programmes. The following is a list of projects that have directly funded the research. There are other projects in which the results of this work have been applied but that are not listed here. 7.1.1 IST-2001-37652 Hard Real-Time CORBA (HRTC) The focus of this project was that of distributed control applications. This type of applications is usually represented by a distributed information system that closes one or more control loops to keep several interacting target systems in a controlled state. The temporal behavior of the control system is critical in this type of applications due to the dynamic effects that arise from delays or jitter produced from the software / hardware execution path needed for control algorithm calculations. Distributed Control systems are software-intensive applications that are becoming extremely complex as more and more functionality is being required from them. Complexity as a real engineering challenge can be successfully addressed by distributed object technology. As already known and presented in this Thesis, one of the leading technologies regarding distributed objects is the OMG’s object request brokering model proposed by the CORBA specification. While CORBA specifications do address real-time issues they deal only with soft real-time systems and that, simply, is not enough for certain types of distributed systems (namely controllers) where timing properties are critical. The construction of distributed CORBA systems where time-related properties are critical was the objective of HRTC. Concretely, the central objectives of HRTC were (i) the analysis and identification of hard-real time requirements posed by CORBA-based distributed control systems, and to develop theory / methodology for hard-real time CORBA applications, (ii) the enhancement of CORBA specifications with corresponding interfaces in order to build distributed control systems that have real-time requirements with hard timing constraints, and (iii) the implementation of CORBA-pluggable real-time protocols in the ICa ORB for experimental tests. The requirements were developed based on partners' experience and on project specific experiments in two testbeds (one in robot control and one in continuous process control). They were used to launch a specification process inside the OMG by 199 means of the preparation of a RFI (Request for Information), a RFP (Request for Proposals) and by fostering the collaboration with other OMG groups in the elaboration of a proposal in response to the RFP. The participant partners in HRTC were mainly academic with the exception of SCILabs Ingenieros from Spain. Three universities provided their expertise in this project, UPM from Spain in the field of distributed objects, the University of Lund from Sweden as expert in control theory of networked systems and, the Technical University of Vienna from Austria with its knowledge of time-triggered architectures. 7.1.2 IST-1999-10251 Distributed Systems and Networks (DOTS) Object Telecontrol The main objective of the DOTS project was to establish an open software model built upon real-time distributed object technologies and emergent telecontrol standards to allow the optimum exploitation of the interoperation capabilities of devices (Intelligent Electronic Devices) and systems in the distributed context of an electric power grid. The project addressed outstanding problems in this context, and its results are of application in any other industrial control system. Real-time and Minimum CORBA, and the, at the time of the project, emerging IEC-61850 standard for control systems in electrical substations were the supporting technologies of DOTS. In DOTS, the promotion of real-time distributed object technologies for teleoperation of control systems and networks and, the contribution to standardization organizations like IEC were among the core project ideas. Also, in a pragmatic approach, development of tools, middleware and the effective operation of a pilot telecontrol system was carried out within the project. Thus, development helped us to demonstrate the validity of the DOTS approach when facing real-world conditions. Several objectives were pursued during the project. Among them, there were the following: • Definition of a general model for distributed telecontrol networks at electric power grids. This model considered CORBA performance versus the hard real-time constraints imposed by the IEC communications model, specifically identifying QoS (Quality of Service) requirements of the station elements. • Mapping of the IEC-61850 Abstract Communications Service Interface (ACSI) [IEC 01b] to CORBA. Both the general model definition and the mapping specification documents were submitted to the involved IEC and the OMG (Object 200 • • • • • Management Group) Technical Committees, in order to contribute to the promotion of IEC-61850 and Real-time and Minimum CORBA standards. Development of a substation configuration tool for the interoperation of devices at the configuration level, according to the IEC-61850 configuration language specification. Development of an operator interface tool for accessing the objects embedded in distributed and heterogeneous devices located at a power substation. This tool made use of commercial non-real-time CORBA middleware and Java language running in a PC platform and interoperated with the real-time ICa ORB embedded in the RTUs. Implementation of base software, mainly selected IEC-61850 objects and functions, to constitute a development framework on top of which communications networks and industrial control systems can be implemented. The IEC standard functions were represented as ICa CORBA objects so they were accessible by CORBA enabled applications like the operator interface tool. Adaptation of an existing real-time ORB (ICa) to be embedded into a RTU at the substation level. Integration of the middleware, software and tools developed within the project into a pilot system located at a power substation belonging to Red Eléctrica Española to validate the selected approach. The departure point of project was the worldwide recognized necessity to ease the interoperation of systems, networks and devices in electricity distribution installations. The necessity arises from the overwhelming amount of protocols and proprietary interfaces present in this context. This necessity was targeted by two approaches in the context of power substations that were in draft status at the time the project began; the American UCA 2.0 architecture, and the IEC-61850 standard. At that time, to the Consortium’s knowledge, and due to the draft status of these standards, no IEC-61850 or UCA 2.0 compliant system had been implemented nor installed. In fact, the pilot system installed during the project was the first CORBA-based running implementation of the IEC-61850. At the same time of the project, preparation of the Real-time CORBA specification, version 1.0, had just finished and several commercial products for real-time distributed objects technology had been developed under CORBA 2.2 specification. However, it was not usual the integration of this middleware in a real-time environment as hard as the control of embedded devices in a power substation distributed network. Although the project initially targeted the specific context of power substations, it was also aimed to overcome general and outstanding problems that arise in communications networks 201 and distributed systems for industrial control, always related to the integration and interoperation of heterogeneous devices, networks and systems of different manufacturers. Distributed telecontrol networks are composed of a wide variety of systems belonging to different categories, depending on their functionality, location and performance requirements: real-time IEDs (intelligent electronic devices) and deeply embedded RTUs (remote terminal units), maintenance and operation terminals and, at the top level, Supervisory Control and Data Acquisition (SCADA) systems located at control centres of utility companies. Communication among all these devices is achieved through the available links at the field site. DOTS provided a clean open way for interoperation among all of them by providing an architecture with the mixed fundamentals of IEC61850 and embedded real-time CORBA. Partners from four European countries were represented in the DOTS project. ELIOP, UPM, REE and SCILabs Ingenieros from Spain, UNINOVA from Portugal, DECANGETRONICS from France and INNOVA from Italy. 7.1.3 ESPRIT Project 22130 – Distributed Information Technology for Strategic Multiobjective Process Control (DIXIT) The project aimed at producing a software and methodological environment targeted at preventing and solving control problems at different layers of intelligent control (operational, tactical and strategic). The strategic situations to deal with focused on safety, environmental and maintenance issues and the software environment consisted of AI problem solving distributed components integrated by means of a flexible and distributed architecture (ICa). Knowledge engineering techniques and software engineering methods, including object oriented technologies were used and combined with distributed architecture paradigms in order to produce cheaper and more efficient software. A basic objective was to provide an industrial competitiveness advantage by improving automation in a reduced cost approach. Cost was reduced by the construction of generic software components aimed at addressing different types of problems at different control layers, supported and integrated by a flexible distributed-objects architecture, and the definition of a methodology. This was the commitment of the DIXIT project partners. 202 The structure of most distributed control systems is hierarchical and multilayered, similar to a pyramid, where abstraction as well as control mechanisms complexity increases in higher layers. DIXIT dealt with problems and situations included in the intelligent control layers; operational, tactical and strategic. In particular, it focused on the strategic level, especially on environmental, safety and maintenance aspects considering them together with other global objectives of the plant, such as production, quality, etc. All these objectives constituted a constraint network where priorities and weights were assigned depending on the problem at hand. From the technical point of view, the DIXIT final product was composed of different software modules: user interface components, problem solving components and architectural components. The problem solving modules performed tasks such as failure detection, diagnosis, monitoring, tactic creation and recommendation, sensor validation, forecasting and trending, alarm and emergencies handling, prevention, optimisation, etc. The technologies used spanned rule-based systems, neural networks, fuzzy logic, optimisation techniques, etc. In order to support system ubiquity, the DIXIT architectural components combined the blackboard paradigm (already proven as a successful approach for intelligent applications in a previous project called HINT) with other paradigms related to distribution, such as an agent architecture based on the CORBA specification (ICa). The consortium was formed by five technical developer organisations (IIC, acting as coordinator, RH&H, DaCapo, UPM and Euristic Systemes) and two users (Repsol Quimica and Lafarge Ciments). Four European countries were represented in the consortium. 7.2 Exemplar Applications of ICa ICa has been tested in a wide span of distributed control applications. The deployed systems vary from huge plant-wide systems running at all levels of the control hierarchy (from capture of data to control strategy definition) to embedded systems designed for hard real-time. The following are some examples of the applications of ICa in the control world. 203 7.2.1 Process Control Testbed This application was developed in the scope of the HRTC project at ASLab facilities in E.T.S.I.I. The main objective of the Process Control Testbed (PCT) application was to identify (mainly hard real time) requirements for distributed control systems and to perform experiments in conditions of system heterogeneity and integration of legacy and proprietary applications. Experiments were done using conventional IIOP and the ICa real-time protocol plug-in developed for TTP. Figure 75 shows the hardware setup for the process control application. Figure 75: Hardware setup for the process control testbed One of the general requirements for this process control system was that it should be representative of a real plant process control system. The process control system mimicked a plant control system, what can be seen as a (possibly hybrid) redundant network where the nodes are monitoring and control elements. Therefore, the process control system was essentially a redundant network where system components are connected. The hybrid characteristic of current industrial control systems, with distinct networks at different levels, was intentionally eliminated to test the viability of a flat network in control environments. On the other hand, since a conventional (non hard 204 real-time, TCP/IP) and a specialized (HRTP, TTP/C) networks were being compared, the heterogeneous features of a conventional DCS were recovered just by joining both networks in further experiments. The designed process control demonstrator was able to comprehend the functionality of both present and future process plant control systems. The idea was to build such a control system using CORBA components and to check whether it was possible to: 1. Perform the tasks that current systems usually do. 2. Accomplish the tasks that future systems are expected to achieve. A progressive approach to the construction of the process control application was devised in order to test features incrementally. This progression was concretized in a series of scenarios (different network configurations) where a set of experiments was performed. The planned / executed experiments were the following. 1. 2. 3. 4. 5. 6. 7. Single control loop. Legacy system integration. Simulation components integration (not performed). Sequence of events generation. Traffic capacity test. Concurrent access. Response to faults (not performed). These experiments were run on the conventional (Ethernet) and specialized networks (TTP). The final experiment (Figure 76) combined both networks through a bridge. From the batch of tests conducted on the process control platform it was concluded that the integration of CORBA in control loops did not pose significant problems to developers. CORBA is an alternative for process control not only were OPC operates but also at lower levels (field level). The main problem for this to happen is the lack of openness of existing DCSs which impedes the access to system functionalities from the CORBA layer. 205 Figure 76: The process control demonstrator 7.2.2 Distributed Robot Control The testbed for distributed robot control was also developed in the frame of the HRTC project at the Automatic Control Department of the University of Lund in Sweden. The objective of the robot control demonstrator was to clarify the motivation for hard real-time CORBA. This experiment aimed to illustrate the timing deficiencies in 206 distributed computing for robot control between a regular ORB and a hard real-time CORBA ORB, namely ICa ORB. Robot control is a good testbed because they are often required to be very flexible so a component-oriented design based in CORBA objects could be a good match for this type of systems. For the experiments setup advantage was taken of the available facilities of Lund’s University. The primary system included an ABB IRB2000 manipulator, which was completely reconfigured by the Swedish team to permit control experiments. The original control computers were removed, hardware interfaces were added, and their own control computers were connected thus forming a completely open controller, including servo control down to the torque control of AC motors, which requires several kHz sampling frequency for each of the six joints. Figure 77 shows the setup for robot experiments. Camera Actuator (motor) Windows PC ETRAX Internet Sensor (resolver) • Linux/RTAI ETRAX • Linux/RTAI PPC * 2: Linux/RTAI & STORK Switch GlobeThrottle • Linux PC Controller Figure 77: Robot control testbed setup The basic purpose of the robot control experiment was to catch a thrown object (a ball) using the 6-DOF industrial robot. The ball trajectory was estimated by means of a stereo vision system and a prediction for the catch point was obtained as input for the robot. The experiment was originally developed (without CORBA hard real-time extensions) as a PhD project at the Department of Automatic Control in Lund. Figure 78 shows a pitch of the ball to the robot. Two wall-mounted cameras provide the stereoscopic vision input for calculation of ball trajectory. 207 Figure 78: The Irb-2000 robot ready to catch the thrown ball. Notice the two cameras on the wall. The hard real-time core of the system depicted in Figure 77 is formed by the four nodes Sensor, Controller, Actuator and the Globe Throttle PC configured to schedule hard real-time network traffic in RT-Ethernet. The Globe Throttle and the switch guarantee that an upper bound for hard real-time communications in the network can be controlled. This is achieved by limiting the traffic bandwidth that each node is able to request from the network. Technically this is called throttling. For this environment, ICa ORB was ported to the RTAI-Linux platform so it could be used to wrap the control APIs of the testbed nodes. 7.2.3 Substation Telecontrol Application The DOTS project was focused towards the development of open control system models based on standards and on an open software architecture built upon real-time distributedobject software technologies. Among the most important results of DOTS there was the implementation model of the IEC-61850 standard (Figure 79), designed for communication networks and systems in 208 substations. Interoperability among multi-vendor control devices was achieved by mapping the IEC standard to a real-time CORBA middleware, ICa ORB. The mapping to CORBA was submitted in the framework of the project for inclusion as part of the IEC61850 standard. The standard specifies a collection of elementary services that can be performed by substation automation systems or SAS. These services such as breaker operation, interlocking, system monitorisation, etc. are offered in the standard by artifacts called Logical Nodes that are distributed physically in different processors and mapped in DOTS to CORBA objects. Figure 79: IEC-61850 standard perspective on function provision based on logical nodes To achieve global behavior in the substation automation system, the distributed logical nodes collaborate to perform Logical Functions. To check the validity of the DOTS approach, a pilot system was developed and installed at a substation of Red Eléctrica Española in Villaviciosa de Odón (Madrid), incorporating the software and tools developed within the project. These tools were the following: • A deeply embedded device (IED or Phasor Measurement Unit, to acquire three voltage and three current measures and to calculate phasors and frequency from them) at the bay level supporting the following IEC-61850 functions: • self-checking, • configuration and maintenance, • control, • protection • and distributed process automation. 209 • A RTU at the substation level, that included the following IEC-61850 functions: • network management, • configuration and maintenance, • alarms and events management, • and station interlocking. • Another RTU that included extended capabilities, like still-alive image acquisition, compression and transmission for surveillance purposes. • An Operator Interface tool for local or remote inspection and interaction with objects distributed in the power substation. • A System Configurator tool, working off-line, to provide interoperability at the substation configuration level. • ICa ORB, the embedded real-time CORBA middleware for communications among the existing distributed objects, using serial and Ethernet links. Through the pilot system (Figure 80) operation, the following key points were demonstrated: • Adequacy of the model for communications networks and systems in power substations and grids. • The flexibility of the power substation objects was not constrained by the communication layout (CORBA versus legacy protocols). Extended functions, like surveillance, not envisaged by current telecontrol protocols, were smoothly integrated into the pilot system. • Dynamic reallocation of the SAS related objects, as well as balance of the processing load among the pilot system RTUs. • Enhanced visibility of the telecontrol objects from operator interfaces or SCADAs located at remote sites, e.g. at a control centre. As a conclusion the project demonstrated that the implementation of IEC-61850 over a CORBA middleware offers extremely good perspectives for complex systems construction and deployment, tasks that are quite more complex using other types of technology. 210 The advantages obtained for maintenance must not be disregarded. CORBA enables running application analysis and monitorisation and simplifies system evolution. Changing platforms (hardware or operating systems) are not problematic for CORBA based applications as it is in CORBA’s nature to provide transparency across heterogeneous system boundaries. Non Real-time Configuration Terminal Operator Terminal Station Network CORBA Monitor CORBA gateway IED-1 Process Network CORBA RTU IED-2 Real-time Figure 80: Setup of the DOTS pilot system 7.2.4 HYDRA-Vision Hydra-Vision relays real-time digital video to operators distributed in a WAN environment. The Hydra-Vision system has been developed upon the ICa ORB framework to provide control-room engineers with real-time video feed from remotely operated power-generation plants. The system is composed of a collection of nodes that generate video streams (usually video cameras in plants) and another collection of nodes that are clients of these streams. These last nodes can be operation stations, supervision stations, security offices, etc. Video and audio streams are generated using remotely-controlled video cameras and microphones and compressed using real-time MPEG compressors based on hardware. These systems are organized into zones that are controlled by a single system. The equipment (controller, MPEG compressor, multiplexers, etc.) corresponding to a zone receives the name of a Local Control Center (CCL). All these elements are built as ICa agents (CORBA objects), offering CORBA services to other agents in an open architecture. 211 Hydra-Vision employs ATM (Asynchronous Transfer Mode) broadband networking as the base communications infrastructure but, for deployment reasons, IP over ATM was used for all network APIs instead of using native ATM APIs that offered better control over resources [Seckin 98]. The classic network service on the Internet (based on IP) is a best-effort service, where packets from a source are sent to a destination, without guarantees of delivery or timing [Borden 95]. Video traffic requires better control over network resources to guarantee video quality at the client side [Pancha 93]. Hydra-Vision applications are built on top of two virtual networks: • • The control network handles all service negotiation between Hydra-Vision agents using ICa technology. This is the network that all Hydra-Vision agents use to exchange information and service requests with all other agents. The video network handles all video and audio traffic. This network employs multicasting to minimize bandwidth use [Ravindran 97], making it possible for different clients to watch the same video stream concurrently. An authorization system built in the control network manages access to systems resources. Remote clients are usually placed in Integrated Control Centers (CCI) and receive MPEG video feeds, which are decompressed by software and displayed in operator stations. Video feeds can also be watched in any other station in the network (even if they are placed in CCLs). The main graphical user interface is shown in Figure 81 and operates in the three different modes explained below. Figure 81: Hydra-Vision graphical user interface 212 • • • Local: Used when a user wants to watch a video feed in a workstation of the CCL where that camera is placed. The user selects the camera, the target subject and the viewing parameters and the system interacts with all the equipment to serve the video feed to the client. The user can operate the camera and watch the video like in a conventional CCTV system. Remote manual: Used when a user wants to watch a video from a camera in a different CCL. The operation is exactly the same, except that the distance between the camera and the viewing screen can be hundreds of kilometers. Remote automatic: This mode is used for normal operation. Hydra-Vision is integrated with the DCS used to operate the plant enabling it to respond automatically to plant events. This mode is used when the DCS signals a special situation in a plant (for example an alarm triggering). Hydra-Vision operates semi-autonomously in remote automatic mode; an ICa agent automatically requests the video system to generate a video stream from a specific target in the plant to a specific display station. For example, an alarm triggered in a transformer will target a camera on that transformer, generate the compressed video stream and display the video stream in the operation station responsible for that alarm. All these events will happen with a delay of some tenths of a second (mainly due to limited speed in the camera positioning system). 213 Figure 82: Controlled equipment and Hydra-Vision network As the number of concurrent users is not determined in advance and to handle priorities between users, it is necessary to employ policies that control the response of systems to remote and / or local requests. This is done to minimize the impact of non-critical video feeds on critical ones. Criticality is determined by the originating cause and priority level of the video feed. The system runs over a network of UltraSparc workstations running Sun Solaris and Pentium based computers running Microsoft Windows NT (to support MPEG compression boards). Other equipment is also managed by the system as audio and video multiplexers, video quad equipment and recording facilities for video streams. Figure 82 shows some of this equipment and a network of CCLs and CCIs. The potential uses of this type of technology are not restricted to safety applications. This system can also be employed in: • Security: Plants without personnel can be targets for thieves or vandalism. This system can be used by security personnel in remote sites as a CCTV using bidirectional audio to pretend that there is human presence in the plant. 214 • • Maintenance: Experienced maintenance personnel are scarce and by using this technology it is possible to send less experienced personnel to plants that can keep contact with the expert using bi-directional audio and video. Videoconferencing: The homogeneous nature of the system (all nodes share the same functionality) also makes it possible to use the Hydra-Vision for videoconferencing. Broadband integrated DCSs for electric utilities are one of the many applications of distributed object technology in control systems engineering. The system presented in this section was deployed in several hydroelectric plants that Unión Fenosa owns in the province of Guadalajara (Spain). 7.2.5 Teleoperation of a Virtual Robot Robotic systems work alone no more, they collaborate. Systems are changing from isolated robots with built-in control to complex systems like manufacturing cells with multi-robot cooperation which are also integrated with higher-level enterprise systems. Other complex robotics systems exist, like teleoperated robots with human supervisory control or humanoid robots. In order to carry out their tasks, different types of sensors populate the robot work area and the data generated by them must be integrated (e.g. cameras, force sensors) at different control levels (e.g. path-planning, man-machine interfaces). The extreme example of these complex robotic systems are the humanoid robot projects like Cog [Brooks 98], Wabian [Hashimoto 98] and P3 and ASIMO [Inoue 98]. In these projects, components for perception, cognition and mechatronics have to be integrated into a single system. To demonstrate the usefulness of ICa also in the field of robotics, the ICa ORB has been applied to the control of a teleoperated robotics system that includes a force-feedback joystick master and a simulation of an anthropomorphic robot arm. A virtual robotic arm has the advantage of freedom of design. Therefore, the virtual robotic arm design was inspired by the anatomy of the human arm in order to resemble a complex instrument. The aim of the design is to preserve the mechanical characteristics of the human arm as much as possible but, at the same time, to enable technical feasibility that allows proving the advantages of distributed-object system development. 215 The design followed a nature-is-best approach and each articulation complex was investigated with respect to its biomechanics. Cadaver studies like [Kapandji 84] and [Helm 92] were useful for this respect although they are rare in literature and the data varies for each individual. As a result, the model is mainly based on structural data of the human arm and on the qualitative description of physiological motion. The manipulator model comprises the shoulder joint, the elbow and forearm joint as well as the wrist (Figure 83). In the experiments carried out, the 7-DOF robot behaves as a slave while it is commanded by master joystick with force-feedback. Figure 83: The simulated anthropomorphic robot arm The master joystick has only 6-DOF while the robot has one more degree of freedom. This problem was resolved by introducing a condition in the system that obliged to keep the robot forearm in a horizontal position. This means that the simulation of the robot arm looks like a human arm manipulating objects placed on a table or flat surface. The tool used to visualize the robot is called KaVis. It is based on Open Inventor software and its purpose is to visualize the motion of kinematic chains. For the teleoperation experiments a heterogeneous hardware and operating system architecture was devised. Heterogeneity is common in teleoperation environments and one of the points that were to be proved is that CORBA is good for system integration. The heterogeneous teleoperation environment was built around a PC running Microsoft 216 Windows NT to which the master joystick was linked. The virtual robot was running in the KaVis visualizer on top of the IRIX operating system in a Silicon Graphics workstation. To make the experiments more complex, a Sun workstation running Solaris was used as the link between the master joystick and the slave robot. The mission of the Sun workstation was to work as controller of the master and slave systems and as a translator between the master and slave coordinate spaces. The controller software in the Solaris workstation samples the state of the master Cybernet joystick and updates the force feedback the joystick user feels. In this software architecture, the master behaves as a CORBA object which is queried by the controller. The controller is client of the master but at the same time is a server whose services are requested by the virtual robot which is a CORBA client. KaVis provides an API for C++ which allowed it to be integrated with ICa for these experiments. The experiments not only demonstrated how system development is improved by the use of ICa but also that multiple robotic slaves that conform to their IDL descriptions could be controlled. This is shown in Figure 84. Figure 84: Multiple systems can be teleoperated with a single IDL interface 217 7.2.6 RISKMAN RiskMan is a risk manager that was developed as a demonstrator of the DIXIT project technology. DIXIT addressed the problems of the higher layers of the control pyramid, mainly the strategic layer. The strategic layer comprises tasks that affect the plant from a global point of view and commands the lower layers to pursue the defined strategy. The objectives of the tactical layer can be classified regarding continuity, maintenance, production quantity, quality, efficiency and safety. The strategic layer performs a continuous follow-up of these objectives in order to achieve them. Several strategic objectives were selected to be built into RiskMan. The RiskMan systems were deployed in the petrochemical complex that Repsol owns in Tarragona. The complex extends over an area of 150 Hectares and comprises a refinery and a chemical complex with 9 production plants. The plants are organized into three different areas according to potential hazard. In a complex like this, the selected features to test were those related to safety and maintenance. Regarding safety, the objectives of the system were to prevent the occurrence of anomalous situations and to assist in dealing with them by making accurate information available to key personnel. For this, the emergency plan developed by Repsol to comply with European, Spanish and local regulations was used and an intelligent system, RiskMan, was developed. RiskMan prevents and detects dangerous or potentially dangerous situations and classifies them according to four categories of emergency depending on its impact on life and property. RiskMan performs its work by using several AI techniques like neural networks, expert systems or fuzzy technology to supply predictions of system behavior. If an abnormal situation is detected the emergency plan is automatically launched providing up to date information to the persons responsible for the execution of the emergency plan and including a role assignment to each of them. Also external communications are established in order to inform competent authorities via automatic phone call, fax or e-mail. For the maintenance objective, it is also needed, in a system like a petrochemical complex, to analyze the impact of the maintenance operations in the safety of the plant before carrying them out. In fact, although some operations may be harmless by themselves, the combination of several of them in the same geographical area can yield very harmful effects. This is why a work permits system is needed in the 218 complex. RiskMan manages the maintenance operations being carried out at a given instant taking into account the personnel and equipment affected by maintenance. With these constraints, it is able to issue new work permits for a certain area. The architectural technology that supports distribution in RiskMan is ICa. The ORB allowed the integration of heterogeneous systems ranging from VAX-based industrial databases to prediction engines running on Alpha workstations or desktop PCs showing the RiskMan user interface (Figure 85) to direct and assign people roles for emergency handling. Figure 85: RISKMAN user Interface 7.2.7 PIKMAC PIKMAC (Process Information and Knowledge Modeling for Advanced Control) is another demonstrator that was developed in the scope of the DIXIT project. Its objectives, also in the strategic layer, were focused towards production quantity, quality and maintenance. The demonstrator was deployed in the cement plant that Lafarge Ciments runs near Nice. 219 For the production quantity objective a Production Performance Synthetic Indicator (PPSI) was developed. The objective of the tool was to continually provide control room engineers with an indication of production performance regarding quantity and cost. To facilitate information sharing among the different layers of the enterprise, the indicator is presented in different forms according to the different plant departments (e.g. administration, management, maintenance, production, process). This helps all the plant people have a common view of the production process. Regarding quality, the main problem for the manufacturing of clinker is the control of some parameters that affect the production process and hence clinker quality. These parameters cannot be obtained in real-time as they can only be known after laboratory tests. In PIKMAC a Quality Deviation Early Detector (QDED) was implemented to predict clinker quality at one stage of the process taking into account parameters that could be collected in a continuous way from previous stages of the process. QDED helps improve process quality as continuous estimations of present clinker quality can be made. Further, based on the data known at a given moment in a stage of the process, the quality of future clinker production can be estimated provided that the remainder parameters of the process remain unchanged. The indicator can also be used to tune parameters of the different production stages given known parameters of previous stages (the composition of the initial raw material for instance). Figure 86: PIKMAC user interface For the maintenance objective, the aim was to provide assistance to operators in a plant that runs almost unattended during the night and weekends. This is called one220 man shift in the plant jargon. For this purpose, another module the Alarm Manager Operator Assistant (AMOA) was developed. AMOA assesses the operator in order to decide whether a given failure should be fixed immediately or not. This decision is not easily taken as the economic optimum of the plant depends on several complex constraints; electricity cost profile for the hours and day to come, cement storage, production plan and availability of spare parts for repairing. In a system like this, multiple different systems must be integrated to gather and to deliver the information needed for module reasoning in a timely manner. The framework selected for this purpose was that of ICa. Among others ICa was used to integrate the following systems (apart from PIKMAC developed systems). • • • • Alarms generated by the PROTOP digital system. The industrial CIM21 database that recorded process parameters. The automatic laboratory analysis. The Incident Report Database where the problems of the plant were stored. Figure 86 shows the user interface of PIKMAC. 221 Chapter 8 Conclusions and Future Work 223 8.1 Conclusions This section presents the conclusions of the work carried out for this Thesis. The results are grouped in different subsections according to their nature. 8.1.1 Specific Thesis Objectives The introductory chapter of this document presented two Thesis objectives which are reviewed in the following items. • • The first one was that of co-developing a real-time CORBA ORB for its adoption in distributed control systems. This objective has been completely fulfilled as ICa ORB is a piece of software that has proven its usefulness in a multitude of distributed control systems from different domains of control. ICa ORB has been employed in large control systems and in embedded systems. Also, it has been employed with success in systems directed by the occurrence of events as well as in systems based on the time triggered paradigm. That way, ICa demonstrates that it is capable of embracing a wide spectrum of control systems which was a fundamental aim of this Thesis; the demonstration that a single technology can be the basic infrastructure of many different types of control systems, solving at the same time the problem of integration suffered by this type of systems. Solving the integration problem is not in itself a simple business and, in the industry, this problem alone can be defined as harsh at a minimum. The second objective was that of introducing a methodology which could ease the work of implementing CORBA Control Systems. The approach of the ICm methodology has been made from the point of view of domain engineering. This has yielded several advantages: a) ICa can be employed as the core technology to build single CORBA Control Systems by the use of application engineering, b) the ICm methodology can be employed with a multiple system scope point of view, being then possible to implement frameworks on top of ICa ORB for the development of families of distributed control systems for a control domain. Which alternative is used depends on the development team and its objectives. 225 The previous items show that the Thesis objectives have been fulfilled. However, many other important initially secondary objectives which are worth being mentioned have also been fulfilled. The following sub-sections present these objectives. 8.1.2 Awareness of Standardization Bodies The demonstration that the ICa technology can solve many problems of distributed control systems is of great value per se but if these findings are done in isolation there is no way to influence the industry to make use of it. For this reason, one of the concerns during these last years has been to expose the ideas behind ICa in as many forums as possible. The main result of this effort of publicly exposing the ideas behind ICa can be found in the Object Management Group. A great amount of work has been employed in injecting the ideas of CORBA Control Systems into the people of OMG. In consequence, sufficient interest has been raised to create a working group inside the OMG with the objective of fostering the availability and suitability of specifications for distributed control systems based on CORBA technology. The group’s name is Control Systems Working Group and belongs to the Real-Time, Embedded and Specialized Systems subgroup of the Platform Technology Committee of the OMG. Figure 87 shows the web page for the working group at the OMG. Figure 87: Control Systems WG 226web page at OMG's site The creation of the working group took place during the HRTC project which included technology dissemination as a primary objective. All the partners, who include UPM (Spain), the Technical University of Vienna (Austria), the University of Lund (Sweden) and SCILabs Ingenieros (Spain), made great efforts to contribute to the dissemination and exploitation activities work package which the author of this Thesis managed along with the hard real-time ORB work package. It is important to embed future developments regarding CORBA Control Systems in a standardization body. This Thesis work has always fostered an approach based in a common interoperable technology which is vendor independent. That is why it is important to let other people collaborate and promote further specifications. 8.1.3 Domain Systems Engineering of Substation Automation The SAS framework has been presented in the previous chapter and examples of its development were presented in Chapter 6 while the ICa methodology was reviewed. Once more, this domain engineering process to build substation automation systems on top of ICa has been done with an integrative approach. The knowledge of the domain was collected from the expertise of industrial partners in the domain and from the IEC-61850 standard for substation automation. Gathering the knowledge only from industrial partners could result in a valuable but otherwise particular vision of the SAS domain. Instead of that, the idea was to build the framework around the IEC61850 standard and, at the same time, to take advantage of the deep knowledge of the domain from industrial partners. This resulted in two main accomplishments: • • A framework based on the IEC-61850 on top of ICa technology was built. A contribution to the IEC-61850 was achieved as the mapping was introduced to Technical Committee 57 of IEC-61850. It has to be said that the framework was the first working implementation of the standard on CORBA technology in the world. 8.1.4 Results for Third Parties Comprehensibly, over these years, we have gained an understanding of CORBA technology that goes beyond of what could be expected from professionals who simply use the technology. Overall, this expertise is very valuable when dealing either with large complex systems or with systems that try to push the technology close to its 227 limits (e.g. hard real-time CORBA systems). That expertise has been recognized by companies that operate in different industrial environments which have requested the assessment of the author for different projects. Some examples follow 13 . • • • Consultancy has been provided to a Spanish engineering company that operates in the military area for the re-implementation of a N.A.T.O. distributed hard real-time system. One of the largest engineering firms in Europe requested assessment on the available interoperability technologies for the implementation of the forthcoming systems of the Single Sky for Europe project. This project is related to air traffic management from Europe’s point of view and not from the current single country perspective. It is a long term effort within a development time frame of more than ten years. One of the world’s biggest engineering companies from Germany requested the implementation of an ORB over their automation platforms for the control of product lines. The existence of this market from real-world companies should be considered as very important. It means that problems do exist; it means that the technology is capable of solving them (at least of making it easier to deal with them) and, it also means that there already is a real practical application of the ideas behind this Thesis. Thus, in this work a contribution of added value to the European Union policies regarding standardization, interoperability and coherence at the global level and, of improvement of living conditions of Europeans by means of ameliorating the development of DCSs has been achieved. Further, it has already contributed to the economic development and growth of European companies as the items in the previous list demonstrate. 8.1.5 Research Support Another result of this work is the long term collaboration between UPM and SCILabs Ingenieros that is manifest in the signature of an open agreement for further collaboration in the development of the ICa broker and its use in control system research projects. 13 Company names are omitted for confidentiality matters. 228 8.2 Future Work The existing work opens a range of possibilities for CORBA Control Systems that can be addressed by further development efforts. Some of the future work has a general cross-domain scope, as that of ICa, whereas other proposals of further work should be carried out in the frame of concrete control domains. 8.2.1 CORBA Control System Services The idea of CORBA Control System Services is that of extending the ICa architecture by means of pre-built services that are useful for many control domains. ICa is only the initial framework upon which many other useful services can be built. Services in CORBA Control Systems would primarily be ICa objects or assets. A distributed control system would be constructed by composition and configuration of services. The reader should remember that ICa is important because it defines a precise way of building distributed control systems. In the same way, CORBA Control Services would aid to the development of DCSs by providing standard pieces to build upon. This results in a more predictable way to build systems and leads to a simpler definition of corporate standards for system development. The services to be built can be further refined depending on the layer of control where they perform their duty. For instance, at the control layer it is possible to think of services for batch control processes, continuous control processes or discrete control processes. Upper layers can be developed and deployed by means of manufacturing services devised for production scheduling and supervision and even the topmost layers can be implemented by the use of business services envisaged for the complete plant operation. As an example, some possible services follow: • Data servers. Many DCSs today are built upon an architecture that relies on data servers for system state distribution. This can be seen for instance in Figure 14 of page 38 where the AVE power substation control system is depicted. The control network interfaces with upper control layers by means of an OPC server that collects and serves process data. The data server is an interface with all PC technology that houses systems like HMI, gateways to other protocols or even PC-based control systems. Although in CORBA Control Systems there is no need of such interface, this type of architecture, based on the data server, decouples the control layer from the rest. A data server could be built on top of CORBA to perform such function. Also, many vendor solutions interoperate with OPC. The ICa data server can also serve as a gateway to this type of systems. For instance, in the traversal approach 229 • • • • • • proposed earlier in this document, the event service of CORBA could be used as the basis for the specification of the ICa data server for distributed control systems. The OMG has already done some OPC-aligned work in this area [OMG 02c], [OMG 03f]. Alarm and event management services. Alarm and event management services can be easily built either on top of ICa ORB or on top of ICa data servers. Alarms or events are simple data that satisfy user-defined alarm conditions. So it is possible to define what is an alarm or an event and what is not. For instance, an alarm of some type is defined when variable A value is X and / or variable B value is Y. The conditions can be made arbitrarily complex. Alarm services must be complemented with those related to logging for system execution tracing purposes. History services. History services are a form of database services where system information is stored for later retrieval. History services are important because they help to make analysis on the plant data. For instance, trend analysis in a power substation can be used to predict future power consumption. Diagnostic services. In very much the same way alarms can be defined, it is possible to define invariants for the diagnosis of system elements. A system can be designed in a way that some pressure measure must be between two given values. The ICa diagnostic service supervises that such values are never exceeded. Maintenance or other kind of alarms could be triggered in the alarm service if the diagnostic service detects faulty conditions. Reporting and analysis services. Information analysis is vital for the production efficiency improvement of the plant. Thus, analysis and reporting services are important as they allow tracking what is happening in the plant. By data analysis, it is possible to determine if there is a chance of improvement and to perform modifications in the plant. Device integration Services. These are also called ICa Probes. Devices are tweaked by manufacturers to interface to protocols of their choice. By developing ICa Probes, it is possible to integrate every device into the ICa network. An ICa probe acts as a gateway, showing on one side a standard ICa interface defined in IDL and on the other the interface available for the device. The mission of the probe is to act as a protocol translator. HMI Services. Every plant is different. Today’s DCSs development suites offer utilities for the design of HMIs in which custom screens for the DCSs can be developed. In general, most of these utilities are built around OPC so a data server gateway could integrate ICa with these tools. However, it is possible to develop a protocol-independent suite for HMI development having 230 IIOP as its primary protocol but open to other protocol plug-ins. It would be a HMI suite built on top of ICa ORB and its open connection architecture. 8.2.2 Development of Domain Specific Control Frameworks In very much the same way a control framework for substation automation systems was developed, it is possible to address other domains of distributed control systems. By the use of the ICa methodology such frameworks can be reasonably built. This section is intentionally presented after the section dedicated to CORBA Control System Services. This is done so because CORBA Control Services have a crossdomain perspective. This is defined to mean that those services are useful for the implementation of specific distributed control frameworks on top of ICa. As already explained in the ICa methodology chapter, deep expert knowledge is needed for the development of such frameworks inside an effort of domain engineering. However, once the subsystems, assets and objects for a domain are achieved, much can be extracted from the objects that model the domain. The author believes there are opportunities in the Spanish and European markets for the development of such frameworks. Three of such opportunities have already been mentioned. In the case of the high speed railway, control technology is bought from foreign companies. In the case of the Single Sky for Europe project, new systems with a need for a very long lifespan and for change and evolution must be developed in the years to come. Finally, in the case of automation, enhancements to the automatic behaviour of networks of PLCs in production lines (e.g. hot-swapping of a PLC) can be achieved with ICa technology. 8.2.3 The ICa IDE The Integrated Control Environment (ICe) is a tool for the construction of ICa-based systems. The current tools provided by ICa are command-line based. The visual feedback and the interaction level provided by such tools are not very high. In order to correct this situation and to improve development efficiency of ICa systems a development environment can be built around ICa. To put an example, such an environment will be something similar to the graphical interface tools that are available in open-source license form for the GCC compiler or the GDB debugger. A step forward these simple GUIs could be made by incorporating rapid application development (RAD) techniques to this IDE such as the inclusion of wizards for the 231 generation of IDL, stubs and skeleton files of predefined systems. This alternative is being explored by our research group in the context of the Eclipse environment and the MDD (Model Driven Development) model of development. This work is starting in the Aslab research group using the Eclipse platform as a framework. 8.2.4 Extending and Maintaining ICa ORB The ICa ORB is the core of the CORBA Control Systems vision. Its maintenance and evolution if crucial to the reminder systems as it is the nucleus of everything. In consequence, a batch of activities should be spawned pursuant to these objectives. Some of them are excerpted in the following items. • • • • Dynamic Scheduling. ICa ORB addresses fixed time scheduling. Dynamic scheduling poses much more complexity to system development as interaction with an on-line scheduler is needed. The implementation of such a scheduler for real-world systems is not a wise choice for most systems. However, it is worth experimenting with this approach in the following years without ever missing the current ability of ICa for fixed priority systems. Transport plug-in developments. The development of ICa ORB plug-ins for industrial protocols is by far a more pragmatic approach to future developments than the inclusion of dynamic scheduling. At this time, systems in the world are built around many different protocols which coexist the better they can. Chapter 2 already showed examples in which many different protocols are used in the same systems. Protocol plug-ins for ICa ORB can take a twofold role. First they can be used to provide integration with systems external to ICa. Secondly, they can be used in such a way that ICa ORB is employed as the interoperability protocol on top of all the other protocols. The second option leads to a pervasive use of ICa ORB as it makes developer interoperability problems disappear. It is the first stage towards the generalized use of CORBA technology for the implementation of distributed control systems. Porting. This a maintenance and evolution work needed periodically. New versions of operating systems are released periodically and porting to new operating systems is ordinary work in software development. To this future work, great benefit can be taken from the ICa ORB architecture in which a specific portability layer has been designed to isolate operating services from the core of ICa ORB functionality. This means porting ICa is a matter of extending the scope of the portability layer to new systems. Asynchronous Method Invocations. The Messaging specification of CORBA introduced a communications model which allows asynchronous 232 • 8.2.5 invocation of requests for the static interface of CORBA. This permits the invocation of two-way requests in an asynchronous way via callback or poller objects. This feature can be interesting for some ICa applications. Automatic Management of Periodic, Sporadic and Aperiodic Tasks. This are extensions to be included in ICa in the context of the proposal presented in Section 5.13 and of the project COMPARE. ICa Pattern Language Although some effort has been done in capturing the patterns underlying the ICa applications it is necessary to continue this work and elaborate the ICa Pattern Language (ICpl) based on the pattern schema described in [Sanz 03a]. 233 GLOSSARY Abstract Class A specialized class used solely for subtyping. It defines a common set of behaviors to be inherited by its subtypes. It has no instances. (Synonymous with Virtual Class in C++) Abstract Data Type A data type defined to model the data characteristics of realworld objects. An ADT provides a public interface via its permitted operations, but the internal representation and implementation of this interface are private. Abstraction The act of concentrating the essential or general qualities of an object or objects. The resulting concept embodies the "essence" of the objects under consideration. Acknowledged Data Transfer The transmission of data from a source endpoint to one endpoint, or, in the case of multicast, more than one endpoint; and the subsequent response indicating the status of the data transmission. Activation Copying the persistent form of methods and stored data into an executable address space to allow execution of the methods on the stored data. Active Agent An object with a self-owned thread of control. Actor An external agent that interacts with an application or system. This is also a model for concurrent programming. In some context this is a synonym of Agent. Address Field, or fields, in a message identifying both the source and / or destination of the message. ADL Architecture Description Language. Agent Description Language. 235 ADT Abstract Data Type Agent Description Language A language to provide specification mechanisms for agent implementation. Analysis The process of developing a specification of what a system does and how it interacts with its environment. API Application Program Interface Application Facilities Common facilities that are useful within a specific application domain. Application Objects Applications and their components that are managed within an object-oriented system. Application Programming Interface The programming interface used to access and control a library or program. Application A program or a set of programs that provides functionality to the end user. Also refers to specific Algorithm(s) implemented in an IED. Architectural Domain A domain for architectures, i.e. an area of knowledge defined by a family of related architectures. Architectural Style A common core design shared by a collection of software architectures. Architecture Description Graphic and textual languages, standards, and conventions Language (ADL) used to represent a software architecture. ADLs are usually related to DSSAs. ADLs are intended to be used for building models during the development of new systems based on existing architectures or architectures in an Architectural Domain. Architecture A high-level description of the organization of functional responsibilities within a system. Many different levels of architectures are involved in developing software systems, from physical hardware architecture through the logical architecture of an application framework. 236 Association 1. An Association is established when a client-server “linkup” is made and is manifested in the communication path established between a client and a server for the exchange of messages. The Association is closed when the client-server link is Concluded or Aborted. Object association. 2. Meaningful links between objects. A person associated with a company creates the concept of employment. Asynchronous Event Event that occurs independently of the execution of an application. Asynchronous Interaction An interaction between schedulable units (processes and/or threads) in which, after a schedulable unit invokes an operation to take part in the interaction, control is allowed to return to the schedulable unit before the completion of the interaction. Asynchronous Message Asynchronous message communication provides the Communication capability for objects to send messages, even without the existence of the receiving object at the instant the message is sent. The receiving object can retrieve messages at its convenience. There is no blocking or synchronization required between objects. Asynchronous message communication is a foundation for constructing true concurrent computing environments. Asynchronous Request A request in which the client object does not pause or wait for delivery of the request to the recipient; nor does it wait for the results. Atomic Broadcast The transfer of a broadcast message that is either guaranteed to be received by all possible receivers or it is not received by any. Atomicity The property that ensures an operation either changes the state associated with all participating objects consistent with the request, or changes none at all. If a set of operations is atomic, then multiple requests for those operations are serializable. 237 Attribute An identifiable association between an object and a value. An attribute A is made visible to clients as a pair of operations: get A and set A. Read only attributes only generate a get operation. A characteristic or property of an object. Usually implemented as a simple data member or as an association with another object or group of objects. Bi-directional transaction A transaction in which a request and possibly data are conveyed from a Client to a Server and in which a response and possibly data are returned to the Client from the Server. Binding The selection of the method to perform a requested service and of the data to be accessed by that method. (also Dynamic Binding and Static Binding) Broadcast A message placed onto a Communication Network intended to be read and acted on, as appropriate, by any device connected to the network. A Broadcast message will typically contain the sender’s address and a Global recipient address. Example Time Synchronising. CCS CORBA Control Systems. Complex Control Systems. Class Hierarchy Embodies the inheritance relationships between classes. Class Inheritance The construction of a class by incremental modification of other classes. Class Member A method or an attribute of a class. Class Method A class method defines the behavior of the class. Such a method performs tasks that cannot or should not be done at the instance level, such as providing access to class attributes or tracking class usage metrics. Class Object An object that serves as a class. A class object serves as a factory. (Factory) Class A pattern that can be instantiated to create multiple objects with the same behavior. An object is an instance of a class. 238 Types classify objects according to a common interface; classes classify objects according to a common implementation. Classification The act of determining which class or type applies to a specific object. Client An object that requests a service from a server object in a client/server relationship. The code or process that invokes an operation on an object. Client/server A relationship between a client that requests services and servers that provide services. This relationship is paralleled in an object oriented environment by message senders and receivers. CNI Communications Network Interface Collaboration 1. Two or more objects that participate in a client/server relationship in order to provide a service. 2. Collaboration is concerned with the interactions between agents in a multiagent system when the whole system is also considered as an agent with certain structure of system's global state. Particularly, it is concerned with the relationships between individual agents' mental structures and internal states and the system's collective mental structure and state. For example, a collaborative model of multiagent systems may contain a model of system's global intention and individual agent's intention, and we can talk about congruence (that is the consistency between an agent's behaviour and the whole system's global goal or intention) and coherence (that is the consistency between an agent's internal state, such as intention, and the system's goal or intention). COM Common Object Model Common Facilities Facilities useful in many application domains and which are made available through Object Management Architectures (OMA)-compliant class interfaces. (Application Facilities) 239 Common Object Model COM. Microsoft standard for object management Common Object Request Broker Architecture CORBA. A specification for objects to locate and activate one another through an object request broker. The CORBA specification facilitates object request brokers from different vendors to interoperate. Communication Controller A Communication Controller is a virtual device within an IED that provides communication services related to clientserver associations. Communications Stack Hierarchy of communications software. For example the 7 Layer stack of the ISO Reference Model where each layer performs a specific functional role in Open Systems Interconnection communication. Component A conceptual implementation notion. A component is an object that is considered to be part of some containing object. Classes, systems or subsystems that can be designed as reusable pieces. These pieces can then be assembled to create various new applications. Sometimes used to refer to software modules with plug and play behavior. Composition The creation of an object that is an aggregation of one or more objects. Compound Object A conceptual notion. A compound object is an object that is viewed as standing for a set of related objects. Concrete Class A class or type that can have instances. (Contrast with Abstract Class). Conformance Test This test verifies that an object communication interface(s) complies with the specified requirements. Conformance A relation defined over types such that type x conforms to type y if any value that satisfies type x also satisfies type y. Connection An association established between functional units for conveying information. 240 Connectionless The transport of a single datagram or packet of information from one network node to a destination node, or multiple destination nodes, without the establishment of a network connection. Connector A n-ary relation defined over components. Usually employed in relation with software architecture modeling and Æ ADLs. Constraint A relational or behavioral restriction or limit. Usually regarded as a property that must always hold true. Constructor A method that is called when a new instance is created. Constructor methods are used to initialize the new instance. Container Class A class designed to hold and manipulate a collection of objects. Context-Independent Operation An operation in which all requests that identify the operation have the same behavior. (In contrast, the effect of a contextdependent operation might depend upon the identity or location of the client object issuing the request.) Contract Defines the services provided by a server, along with the preconditions and post-conditions that apply to the use of those services. CORBA Common Object Request Broker Architecture Coupling A dependency between two or more classes, usually resulting from collaboration between the classes to provide a service. Loose coupling is based on generic behavior and allows many different classes to be coupled in the same way. Tight coupling is based on more specific implementation details of the participating classes and is not as flexible as loose coupling. CRC Cyclic Redundancy Check. A CRC is performed for each frame and the value is included in that frame when it is transmitted. The CRC check calculation may be simple or complex depending on the protocol being used. The CRC 241 value is used by the recipient communication interface to check and if possible correct errors incurred during transmission of that frame. CSMA/CD Carrier Sense Multiple Access/Collision Detection Data Item A single piece of information to be communicated. Data Type A categorization of values, operations and arguments, typically covering both behavior and representation (e.g., the traditional non-OO programming language notion of type). Datagram A datagram contains within the message all the information for transmission without the requirement for establishing a network connection. DCE Distributed Computing Environment DCOM Distributed Component Object Model Æ Component Object Model Deferred Synchronous Request A request where the client does not wait for completion of the request, but does intend to accept results later. Contrast with synchronous request and one-way request. Delegation The ability of a method to issue a request in such a way that self-reference in the method performing the request returns the same object(s) as self-reference in the method issuing the request.(ÆSelf-Reference) The ability of an object to issue a request to another object in response to a request. The first object therefore delegates the responsibility to the second object. Derivation The act of subclassing an existing class to define a new subclass. (Æ Inheritance) Derived Class The class created through inheritance. A derived class inherits the methods and attributes of its superclass(es) and usually adds its own to distinguish its capabilities or services. 242 Design Pattern A pattern that specifies the way to construct something to satisfy some requirements (equilibrating forces) in some context. Design A process that uses the products of analysis to produce a specification for implementing a system. Also the result of this process. Æ Design Pattern Æ Pattern Destructor A method involved whenever an object is ready to be destroyed. It is usually implemented to revise the actions that were performed during initialization, such as recovery of allocated resources. Device A mechanism or piece of equipment designed to serve a purpose or perform a function. Distributed Computing OSF software specification and implementation to support Environment (DCE) development of distributed applications. (Æ Open Software Foundation, Æ Remote Procedure Call) Distributed Object Computing (DOC) A computing paradigm that distributes cooperating objects across a - possibly heterogeneous - network and allows the objects to interoperate as a unified whole. DLL Æ Dynamic Link Library. DOC Æ Distributed Object Computing. Domain A formal boundary that defines a particular subject or area of interest. The HRCT domain is the domain of software intensive, distributed, complex process control. Domain Expert A person who has special skill or knowledge of a particular domain. Domain Model A model (terminology and semantics) that characterize the elements, processes, and relationships within a family of related systems. Domain-specific software architecture An architecture that captures architectural commonality of multiple, related systems, i.e., systems within the same 243 (DSSA) domain. DRE Distributed Real-time and Embedded DSOM Distributed System Object Model. Æ System Object Model. DSSA Domain-Specific Software Architecture. Dynamic Binding Binding that is performed after a request is issued. (Æ Binding) Dynamic Classification Classification of an object at runtime. This implies that an object's classification can change over time. Dynamic Invocation Constructing and issuing a request whose signature is not known until runtime. Dynamic Link Library A dynamically loaded run-time library. Dynamic Object-Based The end-user functionality provided by one or more Application programs consisting of interoperating objects. EAI Enterprise Application Integration Embedding Creating an object out of a non-object entity by wrapping it in an appropriate shell. (Æ Wrapper) Encapsulation The technique used to hide the implementation details of an object. The services provided by an object are defined and accessible as stated in the object contract. (Often used interchangeably with Information Hiding) Enterprise Modeling A technique for modeling an entire business enterprise from the business manager's point of view. An enterprise model is composed of the objects, events and business rules that describe the enterprise. Separate but related business systems can be built from this model to enhance the efficiency and consistency of the operation of the enterprise. Environment The totality of circumstances surrounding an agent or group of agents, especially we can consider the physical and social 244 environment. Physical Environment: The combination of external physical conditions that affect and influence the growth, development, actions and survival of agents. Social Environment: The complex of social and cultural conditions affecting the nature of an agent or a community. ERP ERP (enterprise resource planning) is an industry term for the broad set of activities supported by multi-module application software that help a manufacturer or other business manage the important parts of its business, including product planning, parts purchasing, maintaining inventories, interacting with suppliers, providing customer service, and tracking orders. ERP can also include application modules for the finance and human resources aspects of a business. Typically, an ERP system uses or is integrated with a relational database system. Event A significant change in the environment or the state of an object that is of interest to another object or system. Externalized Object Reference An object reference expressed as an ORB-specific string. Suitable for storage in files or other external media. Factoring The process of extracting the common properties or behavior from a group of objects so that the common elements can be propagated to a common subclass. Factoring eliminates duplication. Factory A concept that provides a service for creating new objects. Fault-Tolerance The characteristic of a system that allows it to handle the loss of a particular component without interrupting normal operations. Field Bus A communications network shared by multiple, communicating, physical nodes. Typical technology in lower layers of process control systems. FMS Fieldbus Messaging Specification. Formal Parameter A named local object used as an argument to an operation. 245 The value of the object (actual parameter) is assigned by the client who runs the method. Frame Format A template for the actual message to be transmitted. It typically defines the Header, Start of frame, Destination address, Source address, length/type of contained data, the actual data, padding bytes if required and some form of CRC data, then end of frame marker. Framework A set of collaborating abstract and concrete classes that may be used as a template to solve a specific domain problem. Function Functions are tasks that are performed in the application by the components of the CCS. Functional Decomposition The process of refining a problem solution by repeatedly decomposing a problem into smaller and smaller steps. The resulting steps are then programmed as separate modules. Functional Interface Interfaces that define the operations invoked by users of an object service. The audience for these interfaces is the service consumer, the user of the service. These interfaces present the functionality (the useful operations) of the service. An Object Service Definition. Gateway A network interconnection device which supports the full Stack of the relevant Protocol and can convert to a non 7 Layer Protocol for asynchronous transmission over Wide Area Networks. Generalization The inverse of the specialization relation. Generic Object An object (relative to some given Object Service) whose primary purpose for existence is unrelated to the Object Service whose interface it carries. The notion is that the Object Service is provided by having (in principle) any type of object inherit that object service interface and provide an implementation of that interface. An Object Service Domain. Generic Operation The concept that an operation is generic if it can be bound to more than one method. 246 Graphical User Interface Any interface that communicates with the user, primarily through graphical icons. GUI Graphical User Interface. Handle A value that identifies an object. HMI Human Machine Interface. Hub A Hub is a Communications Network component providing multiple ports, each interfacing to a separate media link in a star topology. ICa Integrated Control Architecture ICa ORB The Integrated Control Architecture real-time CORBA broker. ICe Integrated Control Environment ICm Integrated Control Methodology ICpl ICa Pattern Language IDL Interface Definition Language. IED Intelligent Electronic Device Implementation Definition Language A notation for describing implementations. The implementation definition language is currently beyond the scope of the ORB standard. It may contain vendor-specific and adapter-specific notations. Implementation Inheritance The construction of an implementation by incremental modification of other implementations. The ORB does not provide implementation inheritance. Implementation inheritance may be provided by higher level tools. Implementation Object An object that serves as an implementation definition. Implementation objects reside in an implementation 247 repository. Implementation Repository A storage place for object implementation information. Implementation 1. The development phase in which the hardware and software of a system become operational. 2. A definition that provides the information needed to create an object and allow the object to participate in providing an appropriate set of services. An implementation typically includes a description of the data structure used to represent the core state associated with an object, as well as definitions of the methods that access that data structure. It will also typically include information about the intended interface of the object. Implicit Invocation A mechanism of invocation in which the invoker does not know anything about the invokee. Usually implemented through a callback registration mechanism in the invoker or a broadcast of an event to all possible invokees. Inheritance The construction of a definition by incremental modification of other definitions. (Æ Implementation Inheritance) Initialization Setting the initial attribute values of a new object. In-Line Method A mechanism that allows the compiler to replace calls to the method with an expansion of the method code. Instance Name An identifier associated with and designating an instance. Instance Variable A variable that contains a value specific to an object instance. Instance An object created by instantiating a class. An object is an instance of a class. A functional unit comprising an individual named entity having the attributes of a defined class and providing defined services. Instantiation Object creation. The creation of an instance of a specified class. 248 Integrability The capability of a system of being used as part of another, bigger CCS system. Interface Definition Language When used in conjunction with an ORB, IDL statements describe the properties and operations of an object. IDL is used to specify the public interface of a CORBA object. Intelligent Electronic Device An IED is any device incorporating one or more processors, with the capability to receive, process or send, data / control from, or to, an external source. Interface Inheritance The construction of an interface by incremental modification of other interfaces. The CORBA IDL provides interface inheritance. Interface Type A type that is satisfied by any object (literally, by any value that identifies an object) that satisfies a particular interface. (Æ Object Type) Interface 1. A shared boundary between two functional units, defined by functional characteristics e.g.- common physical interconnection characteristics, signal characteristics or other characteristics as appropriate, and the provision of a declared collection of services. 2. A description of a set of possible uses of an object. Specifically, an interface describes a set of potential requests in which an object can meaningfully participate. (Æ also Object Interface, Principal Interface and Type Interface) Interoperability 1. The ability of two systems to exchange services. 2. The ability for two or more ORBs to cooperate to deliver requests to the proper object. Interoperating ORBs appear to a client to be a single ORB. Invariant Relation A relation that cannot be changed so long as it has instances. IP Internet Protocol. The TCP/IP standard protocol. IP defines the datagram that provides the basis of connectionless packet delivery. It includes control and error message protocol providing the equivalent functions to Network services, 249 Layer 3, of the OSI Reference Model. ISO International Organization for Standardization LAN Local Area Network Language Binding or Mapping The means and conventions by which a programmer writing in a specific programming language accesses ORB capabilities. Legacy System A previously existing system or application. Leveling The process of grouping information or concepts at various levels of increasing detail. The top-most level is general in nature and each successive level adds more detail until all aspects of the given subject matter have been explained in detail. Life-Cycle Service The Object Life-Cycle Service provides operations for managing object creation, deletion, copying and equivalence. An Object Service Definition. Link 1. Connection between two processing entities. 2. Relation between two objects (a concept). Local Area Network A communications network which typically covers the area within a building or small industrial complex. Log A record, a journal, of chronologically ordered data e.g. Events + Time Tags + Annotations. Managed Object Clients of System Management services, including the installation and activation service and the operational control service (dynamic behavior). These clients may be application objects, common facilities objects, or other object services. The term is used for compatibility with system management standards (the X/Open GDMO specification and ISO/IEC 10164 System Management Function, Parts 1 to 4). An Object Service Definition. Management (MIS) An organized assembly of resources and procedures 250 Information System required to collect, process and distribute data for use in decision making. A computer system for a business or other organization which collects and analyzes data from all departments, and is designed to provide an organization's management with up-to-date information (such as financial reports, inventory, etc.) at any time. Mapping 1. A set of values having defined correspondence with the quantities, or values, of another set. 2. A rule or process, the OO equivalent of a mathematical function. Given an object of one set, a mapping applies its associative rules to return another set of objects. Member Function (Æ Method) MES Manufacturing execution systems. Systems that use network computing to automate production control and process. By downloading "recipes" and work schedules and uploading production results, MESs bridge the gap between business and plant-floor or process-control systems. Message The mechanism by which objects communicate. A message is sent by a client object to request the service provided by the server object. Meta-Model A model that defines other models. Meta-Object An object that represents a type, operation, class, method or object model entity that describes objects. Meta-Type A type whose instances are also types. Method 1. In systems development, a cohesive set of rules, methods and principles used to guide the modeling and development of software systems. 2. Code that can be executed to perform a requested service. Methods associated with an object are structured into one or more programs. Method Resolution The selection of the method to perform a requested operation. MIDL Microsoft Interface Definition Language. IDL used by 251 Microsoft Windows applications (Æ Language) Interface Definition MMS Manufacturing Message Specification - [ISO 9506] Multi-cast A message placed onto the communication network intended for a limited set of recipients. A Multi-cast message will typically contain the sender’s address and an address field defining a limited set of recipient’s addresses. Multiple Classification Ability of an object to belong to more than one type. Multiple Inheritance The construction of a definition by incremental modification of more than one other definition. Object A combination of a state and a set of methods that explicitly embodies an abstraction characterized by the behavior or relevant requests. An object is an instance of a class. An object models a real world entity and is implemented as a computational entity that encapsulates state and operations (internally implemented as data and methods) and responds to requests for services. An object is a self-contained software package consisting of its own private information (data), its own private procedures (private methods), which manipulate the object's private data, and a public interface (public methods) for communicating with other objects. Object Adapter The ORB component that provides object reference, activation and state-related services to an object implementation. There may be different adapters provided for different implementations. Object Attribute A Field, or, a category or value of data that, together with other attributes, specify the services or data values related to the function and performance of an Object. Object Broker Object Request Broker Object Creation An event that causes an object to exist that is distinct from any other object. 252 Object Interface A description of a set of possible uses of an object. Specifically, an interface describes a set of potential requests in which an object can meaningfully participate as a parameter. It is the union of the object's type interfaces. Object Library/Repository A central repository established expressly to support the identification and reuse of software components, especially classes and other software components. Object Management Architecture The generic architectural design proposed by the OMG. The CORBA specification is the first standard of technology for OMA Object Management Group A non-profit industry group dedicated to promoting objectoriented technology and the standardization of that technology. Object Modeling Technique An object-oriented systems development life cycle developed by General Electric. Now being integrated by its developer Rumbaugh into a new method named Unified Modeling Language. Object Reference A value that precisely identifies an object. Object references are never reused to identify another object. Object Request Broker Provides the means by which objects make and receive requests and responses. The middleware of distributed object computing that provides a means for objects to locate and activate other objects on a network, regardless of the processor or programming language used to develop and implement those objects. Object Services The basic functions provided for object lifecycle management and storage such as creation, deletion, activation, passivation, identification and location. Object State The current information about an object that determines its behavior. Object Type A type the extension of which is a set of objects (literally, a set of values that identify objects). In other words, an object 253 type is satisfied only by (values that identify) objects. (Æ Interface Type) Object Wrapper The result of encapsulating a set of services provided by a non OO application or program interface in order to treat the encapsulated application or interface as an object. Object-Based A programming language or tool that supports the object concept of encapsulation, but not inheritance or polymorphism. Object-Oriented Analysis The process of specifying what a system does by identifying domain objects and defining the behavior and relationships of those objects. Object-Oriented Business Engineering A framework and discipline used to effectively model business processes. It involves identifying business objects, processes, structures, rules, policies, organizational structure and authority, location and logistics, technology and applications. Its goal is to produce precise descriptive models of business objects that can be converted into reusable and easily modifiable software components. Object-Oriented Design The process of developing an implementation specification that incorporates the use of classes and objects. It encourages modeling the real world environment in terms of its entities and their interactions. Object-Oriented Any language, tool or method that focuses on modeling real world systems using the three pillars of objects encapsulation, inheritance and polymorphism. Also OO. OMA Object Management Architecture Oneway Request A request in which the client does not wait for completion of the request, nor does it intend to accept results. Contrast with deferred synchronous request and synchronous request. Ontology An ontology is an explicit specification of the structure of a certain domain (e.g. e-commerce, sport, ...). In practical terms this includes a vocabulary (i.e. a list of logical 254 constants and predicate symbols) for referring to the subject area, and a set of logical statements expressing the constraints existing in the domain and restricting the interpretation of the vocabulary. Ontologies therefore provide a vocabulary for representing and communicating knowledge. Open Software Foundation Non-profit standarization organization dedicated to promote open software standards, i.e. OSF/Motif, OSF/DCE and OSF/1. Now has joined forces with X/Open to constitute the Open Group. Operation A service that can be requested. An operation has an associated signature, which may restrict which actual parameters are possible in a meaningful request. Operation Name A name used in a request to identify an operation. ORB Æ Object Request Broker ORB Core The ORB component that moves a request from a client to the appropriate adapter for the target object. OS Operating System OSF Open Software Foundation OSF/1 UNIX-like operating system based on a microkernel architecture. (Æ Open Software Foundation) OT Object Technology Overloaded Operation Multiple methods of the same name, each having a unique signature. This allows the methods of the same name to be invoked with various argument types. Paradigm A broad framework for thinking about and perceiving reality. A theoretical, philosophical model composed of identifiable theories, laws and generalizations used in defining and solving problems. 255 Parallel Processing The simultaneous execution or computation of two or more programs or operations. Parameter Passing Mode Describes the direction of information flow for an operation parameter. The parameter passing modes are IN, OUT and INOUT. Parameters Parameters are data that define the behaviour of functions. Parameterized Class A class that allows users to declare member functions and data members of "Some Type," which can be used as a template for declaring specialized subclasses that supply the "Missing" types. Partition Decomposing a type into its disjoint subtypes. Pattern A pattern describes a problem, a solution to a problem, and when to apply the solution. Patterns may be categorized in several ways; by example as design patterns, business process patterns and analysis patterns. Æ Design Pattern Persistent Object An object that can survive the process or thread that created it. A persistent object exists until it is explicitly deleted. Plug and Play A type of component that needs little modification to be integrated into a system. Pluggable Transport A CORBA transport layer that can be added or eliminated in run-time. Point to Point A dedicated communication link between two nodes only. Pointer A variable that can hold a memory address of an object. Polymorphic Operation The same operation implemented differently by two or more types. Polymorphism The concept that two or more types of objects can respond to the same request in different ways. Portable Object Adapter An object adapter is the primary means for an object 256 (POA) implementation to access ORB services such as object reference generation. An object adapter exports a public interface to the object implementation, and a private interface to the skeleton. It is built on a private ORB-dependent interface. The Portable Object Adapter offers functionality enough to build portable servers. Post-Condition A constraint that must hold true after the completion of an operation. Pre-Condition A constraint that must hold true before an operation is requested. Presentation Layer Layer 6 of the ISO Reference Model. Principal Interface The interface that describes all requests in which an object is meaningful. Private A scoping mechanism used to restrict access to class members so that other objects cannot them. Property An attribute, the value of which can be changed. Protected A scoping mechanism used to restrict access to class members. Protection The ability to restrict the clients for which a requested service will be performed. Protocol A set of rules governing the information transfer within a communications network. Protocols perform Data Link Control by defining frame format(s), timing, error correction and Handshaking. Public A scoping mechanism used to make member access available to other objects. Query An activity that involves selecting objects from implicitly or explicitly identified collections based on a specific predicate. Rapid Prototyping The iterative process of quickly developing a prototype of an 257 application, usually with the aid of specific GUI-building tools. This process is used to help uncover unknown details of the system under consideration, and to build the system in small increments. Redundancy Refers to spare or duplicate functionality that allows a System to continue to operate without degradation of performance in the event of single failure. e.g. a blown fuse. Reference Architecture An architectural description for a family of applications that describes functional components, connections, protocols, and control. A reference architecture generally consists of a partially-specified system composed of generic or abstract Æ Components, that are replaced by real components when the architecture is instantiated for an actual system. Referential Integrity The property that ensures that a handle which exists in the state associated with another object reliably identifies a single object. Relation An object type that associates two or more object types. A relation is how associations are formed between two or more objects. Remote Procedure Call A protocol which allows a program running on one host to (RPC) cause code to be executed on another host without the programmer needing to explicitly code for this. RPC is an easy and popular paradigm for implementing the clientserver model of distributed computing. An RPC is implemented by sending request message to a remote system (the server) to execute a designated procedure, using arguments supplied, and a result message returned to the caller (the client). There are many variations and subtleties in various implementations, resulting in a variety of different (incompatible) RPC protocols. Repository Usually a central location used to store and organize software components and related definitions, rules, etc. (Æ Object Library/Repository) Request An event consisting of an operation and zero or more actual 258 parameters. A client issues a request to cause a service to be performed. Also associated with a request are the results that can be returned to the client. A message can be used to implement (carry) a request and/ or a result. Requirements A document describing what a software system does from a user's point of view. This document is input into the objectoriented analysis process, where it will be transformed into a much more precise description. Responsibility A service or group of services provided by an object; a responsibility embodies one or more of the purposes of an object. Result The information returned to the client, which can include values as well as status information, indicating that exceptional conditions were raised in attempting to perform the requested service. Reuse Reuse is the process of locating, understanding and incorporating existing knowledge, design and components into a new system. Reuse should occur at all levels of system development analysis, design, implementation, testing, documentation and user training. Role A sequence of activities performed by an agent. A portion of the social behavior of an agent that is characterized by some specificity such as a goal, a set of attributes (for example responsibilities, permissions, activities, and protocols) or providing a functionality/service. Typically related to the realization of a pattern or provision of a function. RTOS Real-time Operating System RTU Remote Terminal Unit- typically a station in a SCADA system, an RTU acts as an interface between the communication network and the plant equipment. Scalability The ability of a system to grow without sacrificing performance. The Scalability is the criteria for a universal and cost effective CCS, taking into account the varying 259 functionality, plant sizes and magnitude ranges. Schema A formal presentation with a defined set of symbols and rules that govern the formation of a representation using the symbols. There are many different kinds of schema, including object, event and activity schemas. Security Domain A subset of computational resources used to define a security policy. Self-Reference The ability for a method to identify the target object for which it was invoked. This notion is referred to by the key words "self " in Smalltalk and "this" in C++ or Java. Semantics The meaning - the essence - of the definition. Server The entity that provides a service that can be requested. Server Class A Server Class comprises of the external visible behaviour of an Application process. Server Object An object providing response to a request for a service. A given object might be a client for some requests and a server for other requests. (Æ Client) Service A functional capability of a resource which can be modified by a sequence of service primitives. Service Cycle The complete process -and the time elapsed on it- from the issue of a request till the response to it. Service Primitive Abstract, implementation independent, representation of an interaction between the service user and the service provider. Service A computation that can be performed in response to a request. Signature Defines the parameters of a given operation including their number order, data types and passing mode; the results, if any; and the possible outcomes (normal vs. exceptional) that might occur. 260 Single Inheritance The construction of a definition by incremental modification of one definition. (Æ Multiple Inheritance) Skeleton The object-interface-specific ORB component that assists an object adapter in passing requests to particular methods. Specialization A class x is a specialization of a class y if x is defined to directly or indirectly inherit from y. State The information about the history of previous requests needed to determine the behavior of future requests. State Machine A formal description of the functionality, responses, actions and re-actions, as a series of discrete, linked states, together with the criteria governing the transition from one state to another specific state. State-Modifying Request A request that by performing the service alters the results of future requests. Static Binding Binding that is performed prior to the actual issuing of a request. (Æ Binding) Static Invocation Constructing a request at compile time. Static Member Function In C++, a function declared part of a class declaration. These functions can be invoked independent of any instances of the class. Strong Typing A language characteristic that requires an explicit type declaration for every value or expression. Strong typing makes static binding feasible. Stub A local procedure corresponding to a single operation that invokes that operation when called. Subclass (Æ Subtype) Sub–functions Sub–functions are sub parts of a main function. A sub– function may be shared by more than one main function. Subscribed data Data that a Client has requested to be supplied on a regular 261 basis, or when trigger condition(s) are satisfied. Subtype A specialized or specific object type. Superclass A class that provides its methods and attributes to another class derived from it via inheritance. Switch An active Communications Network device that facilitates the exchange of data between two devices, on different LAN segments, by dynamically connecting the two LAN segments together only as and when data transfer is required. Effectively multiplies the available bandwidth, allowing LAN segments to run in parallel. Synchronous Request A request in which the client object pauses to wait for completion of the request. System Within this document, “System” refers to CORBA Control Systems, other types of system will be identified by their prefix name. System Object Model SOM is IBM’s CORBA-compliant object request broker for (SOM) a single address space architecture. A similar distributed system object model framework exists to allow objects to communicate across address spaces and networks. An implementation of CORBA by IBM. Target Object An object that receives a request. (Synonymous with Server Object) TCP/IP Transmission Control Protocol/Internet Protocol. A suite of protocols which together provide the functionality up to layer 4, of the ISO OSI Reference Model, without exact layer for layer correspondence. Transient Object An object whose existence is limited by the lifetime of the process or thread that created it. Type A predicate (Boolean function) defined over values that can be used in a signature to restrict a possible parameter or characterize a possible result. Types classify objects 262 according to a common interface; classes classify objects according to a common implementation. Type Interface Defines the requests in which instances of this type can meaningfully participate as a parameter. Example: If given document type and product type in which the interface to document type comprises "edit" and "print," and the interface to product type comprises "set price" and "check inventory," then the object interface of a particular document that is also a product comprises all four requests. UCA 2.0 Utility Communications Architecture version 2.0 describes the concepts of standardised models for Power System Objects. UML Unified Modeling Language. Unified Modelling Language (UML) Standardised constructs and semantics for diagrams, including state machines, which are used to describe / specify the functionality of software-intensive artifacts and systems. Unsolicited Data or Unsolicited Message A Message or Data which is supplied to a Client, or Clients, from a Server without the Client(s) subscribing to that data or message, e.g. “Reset”, “Abort”, “Time”. Does not require a Connection to be established. Use Case/Scenario A description of systems functionality. A description of the sequence of actions that occurs when a user participates in a dialogue with a system. It describes the behavior that is invoked by a system function. Value-Dependent Operation An operation in which the behavior of the corresponding request depends upon which names are used to identify object parameters (if an object can have multiple names). Virtual Class (Æ Abstract Class) Virtual Member Function A member function that can be overridden by derived classes in order to implement a general behavior in a specific manner. Dynamic binding is used at run time to determine which of these functions to actually invoke. 263 WAN Wide Area Network - a communication network which typically covers a geographical area i.e. The network linking a Central Control Room to a number of Substations. Weak Typing A language characteristic that does not require an explicit type declaration for each value or expression. Weak typing makes dynamic binding feasible. Wrapper ÆObject wrapper 264 Bibliography [Alarcón 94] Alarcón, M.I., Rodríguez, P., Almeida, L.B., Sanz, R., Fontaine, L., Gómez, P., Alamán, X., Nordin, P., Bejder, H. and de Pablo, E.; Heterogeneous Integration Architecture for Intelligent Control. Intelligent Systems Engineering, Autumn 1994. [Åstrom 97] Åström, Karl Johan and Björn Wittenmark (1997). Computer Controlled Systems. third ed. Prentice-Hall. New York, NJ. [Åstrom 00] Åström, Karl, Isidori, Alberto, Albertos, Pedro, Blanke, Mogens, Schaufel-berger, Walter and Sanz, Ricardo, Eds.) (2000). Control of Complex Systems. Springer. Berlin. [Bass 98] Bass L., Clements P., Kazman R.(1998) Software Architecture In Practice. 2nd Printing. Addison Wesley. SEI Series in Software Engineering. [BBE 02] Ralf Belschner, Josef Berwanger, Christian Ebner et al.; FlexRay: Requirements Specification; BMW, DaimlerChrysler, Robert Bosch, General Motors, and Opel, Germany, 2002. [Bengtsson 04] PerOlof Bengtsson, Nico Lassing, Jan Bosch and Hans van Vliet. Architecture-Level Modifiability Analysis (ALMA). Journal of Systems and Software, Vol 69, no 1-2, pp 129-147, 2004. [Binstock 95] Binstock A., Rex J., (1995) Practical Algorithms for Programmers, First Printing, Addison Wesley. [Bittner 03] Bittner K., Spence I. (2003) Use Case Modeling. Third Printing. Addison-Wesley. [Booch 99] Booch G., Rumbaugh J., Jacobson I. (1999) The Unified Modeling Language user Guide. Fifth edition. Addison-Wesley [Borden 95] Borden, M., E. Crawley, B. Davie and S. Batsell (1995). Integration of real-time services in an IP-ATM network architecture. RFC 1821. Internet Engineering Task Force. [Borland 00] Inprise, VisiBroker For C++ 4.0 Programmer’s Guide. Inprise Corporation 265 [Bosch 91] Robert Bosch GmbH; CAN Specification Version 2.0; Robert Bosch GmbH, Stuttgart, Germany, 1991. [Brooks 98] R. Brooks, C. Breazeal, M. Marjanovic, B. Scassellati, and M. Williamson. The Cog project: Building a humanoid robot. In Int. Workshop on Humanoid and Human Friendly Robots, Tsukuba, Japan, 1998. [Brown 97] A.W. Brown and K. Short (1997) “On components and objects: The foundations of component based development” Proc 5th Int’l Symp. Assessment of software tools and technologies, IEEE CS Press, Los Alamitos, Calif., pp. 112-121. [Brugali 98] David Brugali. From Objects to Agents: Software Reuse for Distributed Systems. PhD Thesis. Politecnico di Torino, 1998. [Burns 97] A. Burns and A. Wellings (1997) Real-Time Systems and Programming Languages Addisson-Wesley [Bus 96] F. Buschmann, R. Meunier, H. Rohnert, P. Sommerlad, and M. Stal, Pattern Oriented Software Architecture. A System of Patterns. Chichester, UK: Wiley, 1996. [CAN] CAN webpage, available at www.can-cia.org [CDK 94] George Coulouris, Jean Dollimore, and Tim Kindberg. Distributed Systems - Concepts and Design. International Computer Science Series. Addison-Wesley Longman, Inc., 2nd edition, 1994. [ControlNet] ControlNet webpage, available at www.controlnet.org [de Antonio 98] de Antonio A., Arquitectura de Control Inteligente: Un enfoque basado en componentes para Sistemas de Control, Tesis Doctoral, E.T.S.I.I. UPM, Madrid, 1998. [Dev] DeviceNet webpage, available at www.odva.org [Dijkstra 68] E. Dijkstra Cooperating Sequential Processes In Programming Languages (Genuys F. Ed.), pp. 43-112, Academic Press. [Doyle 04] Doyle P, Introduction to Real-time Ethernet I, Contemporary Control Systems Inc, 2004. [Doyle 04a] Doyle P, Introduction to Real-time Ethernet II, Contemporary Control Systems Inc, 2004. [Ens 78] P.H.Enslow What is a distributed data processing system. Computer, 11(1):13-21, January 1978 266 [Foundation] Foundation Fieldbus webpage, available at www. fieldbus.org [Gamma 94] Erich Gamma, Richard Helm, Ralph Jhonson and John Vlissides. Design Patterns. Elements of Reusable Object Oriented Software. Addison-Wesley, 1994. [GE 03] GE Power Systems, D25 Substation Controller Fact Sheet, General Electric Substation Automation Solutions, 2003. [Gokhale 97] Anirudhha Ghokale, Douglas C. Schmidt, (1997), Evaluating the Performance of Demultiplexing Strategies for Real-Time CORBA, GLOBECOM’97, Phoenix, AZ., November 1997 [Hanssen 03] Hanssen F., Jansen P., Real-time Communications Protocols: an Overview, IBM Equinox Programme, 2003. [Har 00] Florian Hartwich, Bernd Müller, Thomas Führer et al.; CAN Network with Time Triggered Communication; Robert Bosch GmbH, Germany, 2000, available at http://www.can.bosch.com/. [Harsu 02] Harsu M., A survey on domain engineering. Report 31, Institute of Software Systems, Tampere University of Technology, December 2002. [Hashimoto 98] S. Hashimoto et al. Humanoid robots in waseda university – hadaly-2and wabian. In Int. Workshop on Humanoid and Human Friendly Robots, Tsukuba, Japan, 1998. [Helm 92] Helm, F. v. d., A finite musculoskeletal model of the shoulder mechanism. Journal of Biomechanics 27(5), 1992. [HINT 94] HINT Manual for System Developers. HINT Consortium, 1994. [Honeywell 04] The Experion Platform, PN-04-016-ENG, Honeywell International 2004. [Horst 97] Horst. F. Schaude. Karlsruher Visualisierer. Technical report, Institut für Prozessrechentechnik und Robotik, Universität Karlsruhe, 1997. [HRTC 02] The HRTC Consortium, 2002, HRTC - Annex I – Description of Work [Hunt 98] Hunt C., (1998) TCP/IP Network Administration, Second Edition, O’Reilly & Associates, Inc. [IEC 01a] IEC TC-57, Communication networks and systems in substations - part71: Basic communication structure for substation and feeder equipment – principles and models. Draft Standard IEC 61850-7-1, 2001. 267 [IEC 01b] IEC TC-57, Communication networks and systems in substations - part72: Basic communication structure for substation and feeder equipment abstract communication service interface (acsi). Draft Standard IEC 61850-7-2, 2001. [IEC 01c] IEC TC-57, Communication networks and systems in substations - part1: Basic principles. Draft Standard IEC 61850-1, 2001. [Inoue 98] H. Inoue. A platform-based humanoid robotics project. In Int. Work-shop on Humanoid and Human Friendly Robots, Tsukuba, Japan, 1998. [Jalote 94] Pankaj Jalote. Fault Tolerance in Distributed Systems. Prentice-Hall, 1994. [John 02] John I., Muthig D., Modeling Variability with use Cases v1.0, Report Number 062-02/E, Fraunhofer IESE, November 2002. [Kapandji 84] Kapandji, I.A. and Koebpke. Funktionelle Anatomie der Gelenke. Ferdinand Eneke Verlag, Stuttgart, 1984. [Lerman 93] Lerman S., (1993) Problem Solving and Computation for Scientists and Engineers, Prentice Hall [Liu 00] Jane W. S. Liu, (2000) Real-Time Systems, Prentice-Hall [Martinsson 02] Anders Martinsson, “Scheduling of real-time traffic in a switched Ethernet network”, M.SC Thesis, Department of Automatic Control, Lund University, 2002. [Modbus] Modbus webpage, available at www.modbus.org [Mowbray 97] Thomas J. Mowbray and Raphael C Malveau. CORBA Design Patterns. Wiley, 1997. [Nichols 98] Nichols B., Buttlar D., Proulx J., (1996) Pthreads Programming, First Edition, O’Reilly [Ogata 90] Katsuhiko Ogata. Modern Control Engineering. Prentice Hall, 1990. [OMG 96a] An Overview of the OMA. Object management Group. [OMG 96b] Realtime CORBA. A White Paper. OMG Realtime SIG. Object Management Group, 1996. [OMG 97] UML Notation Guide, version 1.1 (1 September 1997), The Object Management Group, doc. no. ad/97-08-05. [OMG 98a] Common Object Request Broker Architecture and Specification, Ver. 2.2. Object Management Group, 1998. 268 [OMG 98b] Real-Time CORBA. Joint Revised Submission Document Number orbos/1998-12-10, Object Management Group, Needham, MA, U.S.A., December 1998. Available at http://doc.omg.org/orbos/1998-12-10. [OMG 00] Concurrency Service Specification v1.0. Document Number formal/00-06-14, Object Management Group, Needham, MA, U.S.A., April, 2000. Available at http://doc.omg.org/formal/00-06-14. [OMG 00a] Externalization Service Specification v1.0. Available Specification Document Number formal/00-06-16, Object Management Group, Needham, MA, U.S.A., June 2000. Available at http://doc.omg.org/formal/00-06-16. [OMG 00b] Licensing Service Specification v1.0. Available Specification Document Number formal/00-06-17, Object Management Group, Needham, MA, U.S.A., June 2000. Available at http://doc.omg.org/formal/00-06-17. [OMG 00c] Property Service Specification v1.0. Available Specification Document Number formal/00-06-22, Object Management Group, Needham, MA, U.S.A., June 2000. Available at http://doc.omg.org/formal/00-06-22. [OMG 00d] Query Service Specification v1.0. Available Specification Document Number formal/00-06-23, Object Management Group, Needham, MA, U.S.A., April 2000. Available at http://doc.omg.org/formal/00-06-23. [OMG 00e] Relationship Service Specification v1.0. Available Specification Document Number formal/00-06-24, Object Management Group, Needham, MA, U.S.A., April 2000. Available at http://doc.omg.org/formal/00-06-24. [OMG 00f] Trading Object Service Specification v1.0. Available Specification Document Number formal/00-06-27, Object Management Group, Needham, MA, U.S.A., June 2000. Available at http://doc.omg.org/formal/00-06-27. [OMG 01a] CORBA 3.0 New Components Chapters. Updated CCM specification Document Number ptc/2001-11-03, Object Management Group, Needham, MA, U.S.A., November 30, 2001. Available at http://doc.omg.org/ptc/2001-11-03. [OMG 01b] Real-Time CORBA 2.0: Dynamic Scheduling Specification. Final Adopted specification Document Number ptc/2001-08-34, Object 269 Management Group, Needham, MA, U.S.A., August 2001. Available at http://doc.omg.org/ptc/2001-08-34. [OMG 01c] Open Communications Interface. Submission to the Extensible Transport Framework RFP. Document Number orbos/2001-01-05, Object Management Group, Needham, MA, U.S.A., January 2001. Available at http://doc.omg.org/orbos/2001-01-05. [OMG 02] Common Object Request Broker Architecture and Specification. Release 3.0. Object Management Group. Falls Church, USA. [OMG02a] Enhanced View of Time V1.1. Available Specification Document Number formal/2002-05-07, Object Management Group, Needham, MA, U.S.A., May 2002. Available at http://doc.omg.org/formal/2002-05-07. [OMG 02b] Fault Tolerant CORBA. Available Specification Document Number formal/2002-06-59, Object Management Group, Needham, MA, U.S.A., May 2002. Available at http://doc.omg.org/formal/2002-0659. [OMG 02c] Data Acquisition From Industrial Systems v 1.0. Available Specification Document Number formal/2002-11-07, Object Management Group, Needham, MA, U.S.A., November 2002. Available at http://doc.omg.org/formal/2002-11-07. [OMG 02d] Utility Management Systems Data Access Facility v 1.0. Available Specification Document Number formal/2002-11-08, Object Management Group, Needham, MA, U.S.A., November 2002. Available at http://doc.omg.org/formal/2002-11-08. [OMG 02e] Object Collection Service Specification v 1.0.1. Available Specification Document Number formal/02-08-03, Object Management Group, Needham, MA, U.S.A., November 2002. Available at http://doc.omg.org/formal/02-08-03. [OMG 02f] Time Service v1.1. Available Specification Document Number formal/2002-05-06, Object Management Group, Needham, MA, U.S.A., May 2002. Available at http://doc.omg.org/formal/2002-0506. [OMG 02g] Life Cycle Service Specification v 1.2. Available Specification Document Number formal/2002-09-01, Object Management Group, Needham, MA, U.S.A., September 2002. Available at http://doc.omg.org/formal/2002-09-01. 270 [OMG 02h] Naming Service Specification v 1.2. Available Specification Document Number formal/2002-09-02, Object Management Group, Needham, MA, U.S.A., September 2002. Available at http://doc.omg.org/formal/2002-09-02. [OMG 02i] Persistent State Service v 2.0. Available Specification Document Number formal/2002-09-06, Object Management Group, Needham, MA, U.S.A., September 2002. Available at http://doc.omg.org/formal/2002-09-06. [OMG 02j] Security Service Specification v 1.8. Available Specification Document Number formal/2002-03-11, Object Management Group, Needham, MA, U.S.A., March 2002. Available at http://doc.omg.org/formal/02-03-11. [OMG 02k] Minimum CORBA Specification v1.0. Available Specification Document Number formal/2002-08-01, Object Management Group, Needham, MA, U.S.A., August 2002. Available at http://doc.omg.org/formal/02-08-01. [OMG 02l] CORBA Components v3.0. Available Specification Document Number formal/02-06-65, Object Management Group, Needham, MA, U.S.A., August 2002. Available at http://doc.omg.org/formal/02-06-65. [OMG 02m] UML Profile for CORBA Specification v1.0. Available Specification Document Number formal/2002-04-01, Object Management Group, Needham, MA, U.S.A., April, 2002. Available at http://doc.omg.org/formal/2002-04-01. [OMG 02n] Extensible Transport Framework. Joint Revised Submission. Document Number mars/2002-09-06, Object Management Group, Needham, MA, U.S.A., September, 2002. Available at http://doc.omg.org/mars/2002-09-06. [OMG 03] Real-time CORBA Specification v2.0. Available Specification Document Number formal/2003-11-01, Object Management Group, Needham, MA, U.S.A., November, 2003. Available at http://doc.omg.org/formal/2003-11-01. [OMG 03a] Extensible Transport Framework. Revised Submission Document Number mars/2003-02-01, Object Management Group, Needham, MA, U.S.A., March 3, 2003. Available at http://doc.omg.org/mars/2003-02-01. 271 [OMG 03b] Data Distribution Service submission. Joint Submission Document Number mars/2003-03-16, Object Management Group, Needham, MA, U.S.A., March, 2003. Available at http://doc.omg.org/mars/2003-03-16. [OMG 03c] Smart Transducers Interface V1.0. Available Specification Document Number formal/2003-01-01, Object Management Group, Needham, MA, U.S.A., January 2003. Available at http://doc.omg.org/formal/2003-01-01. [OMG 03d] Unified Modeling Language V1.5. Available Specification Document Number formal/2003-03-01, Object Management Group, Needham, MA, U.S.A., March 2003. Available at http://doc.omg.org/formal/2003-03-01. [OMG 03e] Data Distribution Service Final Adopted Specification. Available Specification Document Number ptc/2003-07-07, Object Management Group, Needham, MA, U.S.A., July 2003. Available at http://doc.omg.org/ptc/2003-07-07. [OMG 03f] Historical Data Acquisition From Industrial Systems (HDAIS) v 1.0. Available Specification Document Number dtc/2003-02-01, Object Management Group, Needham, MA, U.S.A., May 2002. Available at http://doc.omg.org/dtc/2003-02-01. [OMG 03g] Lightweight Logging Service v1.0. Available Specification Document Number formal/03-11-03, Object Management Group, Needham, MA, U.S.A., November 2003. Available at http://doc.omg.org/formal/03-11-03. [OMG 03h] Telecoms Log Service v1.1.1. Available Specification Document Number formal/03-06-01, Object Management Group, Needham, MA, U.S.A., June 2003. Available at http://doc.omg.org/formal/0306-01. [OMG 03i] Transaction Service Specification v1.4. Available Specification Document Number formal/03-09-02, Object Management Group, Needham, MA, U.S.A., September 2003. Available at http://doc.omg.org/formal/03-09-02. [OMG 04] Event Service Specification v1.2. Available Specification Document Number formal/04-10-02, Object Management Group, Needham, MA, U.S.A., October 2004. Available at http://doc.omg.org/formal/04-10-02. 272 [OMG 04a] Lightweight Services Specification v1.0. Available Specification Document Number formal/04-10-01, Object Management Group, Needham, MA, U.S.A., October 2004. Available at http://doc.omg.org/formal/04-10-01. [OMG 04b] Notification Service Specification v1.1. Available Specification Document Number formal/04-10-11, Object Management Group, Needham, MA, U.S.A., October 2004. Available at http://doc.omg.org/formal/04-10-11. [OMG 04c] Lightweight CORBA Component Model. As revised by the Lightweight CCM 2003 FTF. Document Number ptc/2004-06-10, Object Management Group, Needham, MA, U.S.A., May 2004. Available at http://doc.omg.org/ptc/04-06-10. [OMG 04d] UML Profile for Modeling Quality of Service and Fault Tolerance Characteristics and Mechanisms. Final Adopted Specification Document Number ptc/2004-09-01, Object Management Group, Needham, MA, U.S.A., September 2004. Available at http://doc.omg.org/ptc/04-09-01. [OMG 05] Real-time CORBA Specification v1.2. Available Specification Document Number formal/05-01-04, Object Management Group, Needham, MA, U.S.A., January 2005. Available at http://doc.omg.org/formal/05-01-04. [OMG 05a] UML Profile for Schedulability, Performance and Time Specification v1.1. Available Specification Document Number formal/05-01-02, Object Management Group, Needham, MA, U.S.A., January 2005. Available at http://doc.omg.org/formal/05-01-02. [Pancha 93] Pancha, P. and M. El Zarki (1993). Bandwidth-allocation schemes for variable-bit-rate mpeg source in atm networks. IEEE Transactions on Circuits and Systems 3(3), 190–198. [Pascual 00] Pascual, J., Charte, F., Segarra, M., de Antonio, A., Clavijo, J., Programacióm Avanzada en Windows 2000 con Visual C++ y MFC McGraw Hill 2000 [Profibus] Profibus webpage, available at www.profibus.com [Ravindran 97] Ravindran, K. and T.J. Gong (1997). Resource allocation control protocols for multicast data transport. In: IEEE 6th International Conference on Computer Communications and Networks. Las Vegas, USA. pp. 22–25. 273 [Rodríguez 99] Rodríguez Hernández M., Modelado de Sistemas Industriales Complejos, Tesis Doctoral, E.T.S.I.I., UPM, Mayo, 1999. [Sanz 00] Sanz, Ricardo (2000). Agents for complex control systems. Chap. 10, pp. 171–190. In: Automation, Control and Complexity. an Integrated Approach, Eds. Samad and Weyrauch, John Wiley & Sons. 2000. [Sanz 91] Sanz, R., A.Jiménez, R.Galán, F.Matía and E.A.Puente. Intelligent Process Control: The CONEX Architecture. In Engineering Systems with Intelligence. S. Tzafestas (Ed.). Kluwer Academic Publishers, 1991. [Sanz 94] Sanz, R., R.Galán, A.Jiménez, F.Matía, J.Velasco and G.Martínez. Computational Intelligence in Process Control. ICNN'94, IEEE International Conference in Neural Networks. Orlando, USA, 1994. [Sanz 96] Sanz, R., F.Matía, R.Galán and A. Jiménez. Integration of Fuzzy Technology in Complex Process Control Systems. FLAMOC'96. Sydney, Australia, 1996. [Sanz 99a] Sanz, Ricardo, Idoia Alarcón, Miguel J. Segarra, Angel de Antonio and José A. Clavijo (1999a). Progressive domain focalization in intelligent control systems. Control Engineering Practice 7(5), 665– 671. [Sanz 99b] Sanz, Ricardo, Miguel J. Segarra, Angel de Antonio and José A. Clavijo (1999b). ICa: Middleware for intelligent process control. In: IEEE International Symposium on Intelligent Control, ISIC’1999. Cambridge, USA. [Sanz 99c] Sanz, Ricardo, Miguel J. Segarra, Angel de Antonio, Fernando Matía, Agustín Jiménez and Ramón Galán (1999c). Patterns in intelligent control systems. In: Proceedings of IFAC 14th World Congress. Beijing, China. [Sanz 03] Sanz, Ricardo and Janusz Zalewsky. Control Engineering using Design Patterns. IEEE Control Systems Magazine, June 2003. [Sanz 03a] Sanz R., Yela A., Chichilla R., A Pattern Schema for Complex Controllers, 9th IEEE International Conference in Emerging Technologies and Factory Automation, Lisbon, Portugal, September 2003. [Seckin 98] Seckin, Gamze and Forouzan Golshani (1998). A new model for multilayer real-time video transmission for ATM networks. In: 274 Proceedings of the 7th International Conference on Computer Communications and Networks. Lafayette, USA. [Selic 94] Bran Selic, Garth Gullekson and Paul T. Ward. Real-Time Object Oriented Modelling. Wiley, 1994. [Selic 96] Bran Selic & Garth Gullekson. Design Patterns for Real-Time Software. Embedded Systems Conference West ’96. 1996. [Selic 98] Bran Selic and Jim Rumbaugh. Using UML for Modeling Complex Real-Time Systems. March 11, 1998 [Schmidt 95] Douglas C. Schmidt, 1995, Using Design Patterns to Develop Reusable Object-Oriented Communication Software, CACM October’95 [Schmidt 98] Douglas C. Schmidt, David L. Levine and Chris Cleeland, 1998, Architectures and Patterns for Developing High-Performance, Real-Time ORB Endsystems. [Schmidt 97] Schmidt D. C. (1997) Acceptor and Connector: Design Patterns for Initializing Communication Services in Pattern Languages of Program Design (R. Martin, F. Buschmann, and D. Riehle, eds.), Reading, MA: Addison-Wesley. [Schmidt 00] D. Schmidt and F. Kuhns, An Overview of the Real-Time CORBA Specification, IEEE Computer, June 2000 [Sha 93] Sha, L. and Sathaye, S.S. A Systematic Approach to Designing Distributed Real-Time Systems. IEEE Computer, Vol.26, No.9, 1993. [Sha 95] Sha, Lui; Rajkumar, Ragunathan; Gagliardi, Michael. A Software Architecture for Dependable and Evolvable Industrial Computing [Sheldon 95] Sheldon T., (1995) LAN Times Guide to Interoperability, First Edition, Osborne McGraw-Hill. [Sommerville 95] Sommerville I., (1995) Software Engineering, Fifth edition. AddisonWesley [Stroustrup 90] Stroustrup B., Ellis M., (1990) The Annotated C++ Reference Manual, Addison-Wesley [Stroustrup 97] Stroustrup B., (1997) The C++ Programming Language, 3rd Edition, Addison Wesley. [Stallings 97] Stallings W., (1997) Data & Computer Communications, 5th Edition. Prentice Hall, Inc. 275 [Tanenbaum 95] Andrew S. Tanenbaum. Distributed Operating Systems. Prentice Hall, Inc., Englewood Cliffs, New Jersey 07632, 1995. [Tanenbaum 97] Tanenbaum C., (1997) Computer Networks, 3rd Printing. Prentice Hall, Inc. [Tanenbaum 97a] Tanenbaum A., Woodhull A., (1997) Operating Systems Design and Implementation, Second Edition, Prentice-Hall International, Inc. [Temple 03] Temple C., FlexRay Protocol Overview, FlexRay 4th International Workshop, Detroit, U.S.A., March 2003. [Trigaux 03] Trigaux JC., Heymans P. (2003) Modelling variability requirements in software product lines: A comparative survey. PLENTY Project. [TTP 02] TTTech, Time Triggered Protocol TTP/C High Level Specification Document, Specification Edition 1.0.0, 4th July 2002 [TTTech 05] TTech, TTP by-wire Box-Automotive Control Solution for Brushless DC Motors, TTTech Computertechnik AG, 2005. [Wooldridge 00] Michael Wooldridge, Nicholas R. Jennings and David Kinny. The Gaia Methodology for Agent-Oriented Analysis and Design. Autonomous Agents and Multi-Agent Systems, 3, 285-312, 2000. [World] World FIP webpage, available at www.worldfip.org 276 277