On October 13, 2003, IBM and the Chinese Ministry of Education announced in Beijing that the two sides will establish a Chinese education and research grid to promote comprehensive cooperation among universities across the country in education, research and broader projects. The project was jointly proposed by 12 universities including Peking University, South China University of Technology, and Tsinghua University. It is the most ambitious grid project launched by the government so far and one of the largest grid computing projects in the world. Applications include life sciences, image processing, and distance education. By the time the grid is built, it will share the resources of 100 national 211 engineering construction key universities on the education and research network, and will achieve more than 15 trillion floating-point operations when the project is completed. The Forbes ASAP, the scientific version of Forbes magazine, predicted in 2001 that the next wave of the Internet would be the World Wide Web upgrade to the Great Global Grid. So what exactly is a grid?
1 Grid meaning
1.1 Grid Concept
A grid is an integrated computing and resource environment, or a pool of computing resources. It can integrate the entire Internet into a huge supercomputer, enabling a wide range of related computing resources, storage resources, data resources, information resources, knowledge resources, expert resources, equipment resources and even talents. Comprehensive sharing of resources. The fundamental feature of the grid is resource sharing, eliminating resource islands. Of course, we can also construct regional grids, enterprise internal grids, LAN grids, and even home grids and personal grids. The grid concept can be understood from the following three aspects:
First, conceptually, the goal of the grid is to share resources and distribute collaborative work. This concept of grid can clearly guide the resources of various departments in the industry and enterprises for the unified planning, deployment, integration and sharing of the industry or enterprises as a whole, not just the various parts of the industry or large enterprises to plan and possess themselves. And use resources.
Second, the grid is a technology. In order to achieve multiple types of distributed resource sharing and collaboration, network computing technology must address multiple levels of resource sharing and collaboration technologies, develop grid standards, and elevate the Internet from a platform for communication and information interaction to a platform for resource sharing. However, current technologies such as parallel computing and distributed computing middleware are far from solving the problem of resource sharing among multiple organizations, and the key problems faced by grid computing such as joint processing and computing between multiple systems in a wide-area domain. Therefore, the research of grid computing technology is unique, urgent and challenging.
Third, the grid is the infrastructure, which is the basic facility for integrating resources such as computers, data, equipment, and services through various grids. The establishment of this kind of facility will enable users to use the power provided by the grid as easily as we need power on demand at the user end without having to install a large number of complete computer systems and complex software on the user side. In this way, equipment, software investment and maintenance overhead will be greatly reduced.
figure 1
1.2 Grid composition
The build hierarchy of the grid environment is shown in Figure 1. It consists of resources, middleware, tool software, and applications. The resources are composed of various resources distributed on the Internet, including various types of hosts, workstations, and even PCs. They can also be cluster systems, large storage devices, databases, or other devices of the above models. Middleware is the core of grid computing and is responsible for providing remote process management, resource allocation, storage access, login and authentication, security, and quality of service (QoS). Tools and applications provide the environment, tools, languages, and interfaces that users use for secondary development to better utilize grid resources.
1.3 Grid criteria
To test whether a system is a grid, fundamentally, it must look at the application, business value, and scientific conclusions that the system can provide, rather than its system structure. Ian Foster, a neighbor of Global Grid Research, has three limits for grids:
First, coordinate non-centralized control of resources. The grid integrates various resources and coordinates various users. These resources and users are in different control domains, such as personal computers and central computers, different management units of the same or different companies; the grid also addresses security, policies, usage fees, membership in such distributed environments. Permissions and other issues. Otherwise, it can only be called a local management system rather than a grid.
Second, use standard, open, common protocols and interfaces. Grids are built on versatile protocols and interfaces that address basic issues such as authentication, authorization, resource discovery, and resource access. Otherwise, it can only be a specific application system rather than a grid.
Second, use standard, open, common protocols and interfaces. Grids are built on versatile protocols and interfaces that address basic issues such as authentication, authorization, resource discovery, and resource access. Otherwise, it can only be a specific application system rather than a grid.
Third, get non-trivial quality of service. The grid allows its resources to be coordinated to achieve multiple quality of service to meet different user needs, such as system response time, throughput, availability, security, and resource relocation, making the combined system more efficient than its parts. The sum of the effects is much larger.
2 Grid architecture
At present, there are two important grid architectures: one is the five-layer hourglass structure proposed by Ian Foster earlier; the other is under the influence of the industry represented by IBM. After considering the development and influence of Web technology, OGSA (Open Grid Services Architect) proposed by Structured Web Service such as Ian Foster.
2.1 Five-layer hourglass structure
The five-layer hourglass structure is a structure with a wide range of influences. Its main feature is simplicity. It mainly focuses on the description of positioning rather than the specific protocol definition. The basic idea is to focus on the "protocol" and emphasize the importance of API (ApplicaTIon Programming Interfaces) and SDK (Software Development Kits).
The five-layer hourglass model starts from the bottom layer as the structural layer, the connection layer, the resource layer, the convergence layer, and the application layer.
The grid construction layer is composed of various physical resources, including storage resources, computing resources, directories, databases, network resources, sensors, etc. The basic function of the construction layer is to control and manage local resources, and provide an interface for accessing these resources upwards. .
The grid connection layer implements communication and data exchange between the construction layer resources, and defines the core communication and authentication protocols.
The grid resource layer is built on the communication layer and communication protocol of the connection layer, providing services such as data access, computer access, status and performance information access. It considers a single local resource, and the global state and atomic operations across the set of distributed resources are considered by the aggregation layer.
The main function of the grid aggregation layer is to coordinate the coexistence of "multiple" resources and coordinate tasks. The aggregation layer implements more advanced applications on a resource basis. The aggregation layer can be divided into a common aggregation layer and an aggregation layer for specific problems.
The grid application layer exists in the virtual organization environment. The application can be constructed according to the service defined above at a level. It can invoke the service of the resource layer or call the service of the convergence layer to meet the application requirements. Taking the power system as an analogy, the first four levels are equivalent to power plants, power grids, substations, and power distribution rooms, while the application layer is equivalent to the electric gates, electricity meters, and power outlets in the home.
Another important feature is the hourglass shape, as shown in Figure 2. The core protocol forms a bottleneck in the protocol hierarchy. The resource layer and the connection layer together form the core bottleneck of the core, which provides secure access to resources. 
2.2 Open Grid Service Architecture OGSA
The open grid service system OGSA is a framework consisting of nodes and connections. The nodes of the framework are grid services and the connections between grid services are the languages ​​used by the grid services to communicate with each other. Grid services are special network services designed to maintain and manage the grid system.
The OGSA grid is also a five-layer structure with the same structure as the five-layer hourglass structure. The bottom-up is the structural layer, the connection layer, the resource layer, the convergence layer, and the application layer. However, the OGSA structure has the following characteristics compared to the five-layer hourglass structure:
(1) Service-centric model
If the five-layer hourglass structure is a protocol-centric "protocol structure" that attempts to achieve the sharing of resources, OGSA is a service-centric "service structure" that implements the sharing of services. OGSA sees everything as a service and defines a "grid service" that provides a set of interfaces that explicitly follow specific conventions to address issues such as service discovery, dynamic service creation, lifecycle management, and notifications. Therefore, a grid is a collection of scalable grid services. Simply put, Grid Service = Interface / Behavior + Service Data.
(2) Unified Web Service Framework
Web Service describes a new and important distributed computing paradigm that defines a technique for describing the software components being accessed, the methods for accessing components, and finding relevant services to discover methods, solving discovery and stimulating The problem of permanent service. OGSA is a standards-compliant Web service framework. However, in the grid, a large number of temporary services, OGSA extended the Web service, proposed the imitation of the Grid Service, so that it can support temporary service instances, and can create and delete pillars.
(3) Breaking through the field of technology application
Just as Web technology first emerged as a scientific agreement, but later in the commercial field, OGSA transferred the grid technology originally used in the field of technology to the industrial and commercial field. The OGSA-oriented service feature allows us to virtualize resources at different levels, so the same mechanisms and abstractions can be applied to the collaboration of distributed grid support across multiple organizations, or across multiple features of the primary environment.
2.3 Application example: Globus system
Globus is a network computing project developed by the Argonne National Laboratory in the United States, with 12 universities and research institutions participating in the project. Globus researched key theories of network computing such as resource management, information security, information service, and data management, and developed network computing tool software (Toolkit) running on various platforms to help build and plan large-scale network test platforms and develop large-scale networks. Application software running on the network system. Toolkit is the most important result of Globus, and its first version was launched in 1999. On January 13, 2003, the OGSA-compliant Globus Toolkit 3.0 (Alpha version) was released at the first Globus world conference. This marks that OGSA has moved from a concept, an architecture, to a stage of practice. Toolkit is open source and anyone can download the source code directly from its website.
Globus's protocol is divided into five layers: the construction layer, the connection layer, the resource layer, the collection layer, and the application layer. Each layer has its own service, API and SDK, and the upper layer protocol calls the services of the lower layer protocol. Global applications within the grid invoke the operating system through the services provided by the protocol. Globus's grid computing protocol is based on Internet protocols and is based on communications, routing, name resolution and other functions in the Internet Protocol. According to Globus, existing sharing schemes, such as the Internet, B2B, ASP, SSP, Java, CORBA, DCE, etc., either do not fully meet the needs of virtual organizations in terms of the flexibility of shared configuration or the type of shared resources. At the same time, Globus does not attempt to replace the existing technology, but rather hopes to establish a higher level of sharing on top of the existing technology. To effectively support the grid computing environment, the Globus toolkit provides a range of services, software libraries, programming interfaces (APIs), and usage examples for the various protocols proposed in the Globus project.
To date, the Globus Toolkit has become the de facto grid standard. Some important companies, including IBM, Microsoft, Compaq, Cray, SGI, Sun, Fujitsu, Hitachi, NEC, etc., have publicly announced their support for the Globus Toolkit. At present, most grid projects are built based on the protocols and services provided by the Globus Toolkit, such as the US physical grid GriPhyN, the European data grid DataGrid, the Dutch cluster computer grid DAS-2, and the US Department of Energy. Science Grid, DISCOM Grid, TeraGrid in American academia, etc.
3 Grid research history and current situation
From the developed countries of the United States, Japan, and Europe to developing countries such as India, large-scale grid research programs have been launched and have received strong support from the industry. The development of the grid can basically be divided into the following stages:
The first is the germination stage: in the early 1990s, it was mainly the test bed of Gigabit network and the implementation of some meta-computing.
The second is the early experimental stage: in the mid-to-late 1990s, such as the I-WAY project, it also includes some academic software projects, such as Globus, Legion, etc.
The third is the rapid development stage: since 2002, there have been a large number of application communities and projects, the development and use of major basic local dialects, and the industry's interest in grid computing is growing, such as IBM, Platform, Microsoft, Sun, Compaq, etc. s company. At the same time, there has also been a significant technical foundation, such as the Globus Toolkit, which has formed the GGF (Global Gria Forum) organization with considerable scale and world influence.
Currently, IBM is a leading provider of grid systems and services, and has provided products and services to grid systems for many technology groups, government agencies, and commercial users, including the UK National Grid, the Netherlands, and North Carolina. State bio-grid and more. In 2002, Sun Microsystems Inc. released a beta version of the "Grid Engine" Enterprise Edition. HP also proposed the UTIlity CompuTIng program and the UTIlity Data Center product. Oracle introduced the Globus-based database application tool for grids in November 2002. The computer information processing system of the 2008 Beijing Olympic Games will use the grid, and IBM, Oracle, SUN, and NP have launched a series of products that can be applied to the grid. In May 2003, the Ministry of Education, Culture, Sports, Science and Technology decided to invest 70 billion yen to develop a super-large grid computer. It appears to be nearly 10 times faster in the world's fastest computing computer, and will reach 300 trillion times per second.
China's "10th Five-Year Plan" 863 Program's high-performance computing projects and software projects and other major scientific research projects are related to grid technology. At present, China has carried out two projects of “National High Performance Computing Environment†and “Advanced Shanghai Computational Infrastructure Beijing and Shanghai Pilot Project†to study grid computing. The Vega Project (Vega Project) underway by the Institute of Computing Technology of the Chinese Academy of Sciences is the study of grid computing with metadata, component framework, agent, grid public information protocol and grid computing protocol as the main breakthrough points.
4 Grid problems and development prospects
4.1 There is a problem
Next-generation Internet technology is completely new, with revolutionary changes from fiber to routers, switches, upper-layer servers, operating systems, and various system software and applications. Thus, Globus and its various alternatives will face significant obstacles in the development of the grid. In order to achieve a wide range of applications, the following issues must also be addressed:
(1) Standards are the key to success. Just as the TCP/IP protocol is at the heart of the Internet, building grid computing also requires defining standard protocols and services. So far, there is no formal standard for grid computing, but in terms of core technology, GlobusToolkit has become the de facto standard for grid computing.
(2) Dynamic allocation of grid resources. How to achieve coordinated resource sharing and collaboration among dynamic, heterogeneous virtual organizations is a very important issue in the grid. The resource allocation techniques of some existing parallel and distributed computing systems are not well adapted to the characteristics of computing grid resource allocation problems.
(3) Delays caused by data passing through the Internet. Intelligent software should ensure that data is transmitted on time, otherwise the means of mesh processing problems will be limited to "parallel operations." Parallel computing is done on different machines, and one computer does not have to wait for the processing results of another computer.
(4) At present, the problem of insufficient data transmission capability of the Internet. To this end, the development grid will be combined with the construction of next-generation broadband Internet (such as the "Next Generation Internet (NGI)" and "Internet2, etc." in the United States. On the other hand, the use of wireless mobile and satellite communications is also a realistic approach.
(5) Further solve the problem of human-machine integration, and make the network grid more personalized, intelligent and scientific.
(6) Solve the problems of intellectual property rights, mutual trust and compensation in online resource sharing through legal means and how to ensure the security, authentication and reliability of grid computing.
4.2 Development prospects
Grid is a problem-oriented and application-oriented technology. Its goal is to achieve high-performance resource sharing and collaborative work on the network virtual environment, eliminating information silos and resource islands. This new network system will not only bring about revolutionary and structural changes in the acquisition, distribution, transmission and effective use of information resources, but will fundamentally change our research methods, educational methods, lifestyles and production. The way of activity. Once the grid is built, a lot of work and research that could not be done before can be carried out, which can drive the production and consumption of a large number of related products in the industry. The grid-based will stimulate countless service provision and service consumption. According to Forbes ASAP, grid technology will reach its peak in 2005 and bring a new life to the Internet. If grid technology can drive the market to grow at the expected 17% annual growth rate, then in 2020 a large industry with an annual output value of $20 trillion will be formed.