forked from hpc/OpenLorenz
Web-based HPC dashboard and more
License
Bhaavya/OpenLorenz
This commit does not belong to any branch on this repository, and may belong to a fork outside of the repository.
Folders and files
Name | Name | Last commit message | Last commit date | |
---|---|---|---|---|
Repository files navigation
Welcome to OpenLorenz, the open source version of the Lorenz suite of tools developed at Lawrence Livermore National Laboratory. The goal of Lorenz is to use modern web technologies to make HPC easier. Two of the tools in the suite are available in OpenLorenz -- the dashboard and the job management tool. The dashboard provides a customizable, personalized view of what's going on in the HPC center. The job management tool helps users submit and manage their batch jobs. Deploying Open Lorenz out-of-the-box (using the hardwired 'fake' data) ====================================================================== Apache httpd.conf mods: -------------------------------------------------- Inside the relevant <Directory /> tag... Options ExecCGI Also, uncomment if necessary: AddHandler cgi-script .cgi Perl dependencies: -------------------------------------------------- Install with 'cpan' utility preferably, with dependencies enabled: Class::Delegator Date::Format HTML::Template HTTP::BrowserDetect JSON Redis Modify paths in the OpenLorenz code to match your site ------------------------------------------------------ Makefile (Fix 2 variables) mylc/mylc.cgi (Fix path in begin block) lora/lora.cgi (Fix path in begin block) server/lib/perl/Lorenz/DependencyManager.pm (Fix 3 paths) Then... -------------------------------------------------- untar into htdocs/lorenz make fixperms try loading: http://<yourserver>/lorenz/mylc/mylc.cgi Porting Overview ============================= Once the default OpenLorenz instance is running, turning the fake data into real data from your computer center is the next task. One finds wide variability in how authentication, user data, resource management, and all the other key information associated with a computer center are handled. Lorenz was built to operate within the Livermore Computing center at LLNL, but is structured to support porting to other sites. Lorenz features a number of APIs, including a REST API upon which the web services are built. The REST API is how the client (browser-based) code interacts with the web server and ultimately the HPC resources. This API is called LORA, for LOrenz Rest Api, and is conceptually based on the NERSC Web Toolkit, NEWT. OpenLorenz is shipped with stubbed-out versions of these endpoints, which return properly formatted but fake (hard-wired) data. This gives the porter an idea of the expected data for each web service. The main tasks for porting are: (1) Go through all of the pertinent REST endpoints used by the dashboard and job management tool, and replace the fake data with code that collects real data from your site. In particular, this will include server/lib/perl/Lorenz/*.pm and server/lib/perl/Lorenz/REST/*.pm. See the technical report "Lorenz APIs and LORA REST Services" for more details. (2) For each portlet in the dashboard, modify the displayed data and labels to match what is used at your site. Web Server Dependencies ======================= At LLNL, Lorenz makes use of an Apache web server that is tightly integrated with the rest of the Livermore Computer center. The web server shares a security infrastructure with the HPC resources, meaning that CGI scripts run on the web server have the ability to access other services and clusters on behalf of the user. This is done by using an Apache module that creates Kerberos credentials that are cached for each user when they authenticate with their one-time password. The suexec module executes cgi scripts as each individual user, and the scripts are run with the user's security credentials. Lorenz server code can then make LDAP queries or ssh directly to HPC resources as the user, and launch commands, retrieve data, and perform other actions for the user. While suexec and Kerberos are not requirements for Lorenz to function at your site, you do need to find a way to do something equivalent _if_ you want to perform actions on behalf of the users. If you only want to show generic status and other information in the dashboard, it's not necessary to integrate the web server into your security and cluster environment. Another useful capability is to have a storage area for each user. At LLNL we do this by mounting the users' home directories on the web server, so that Lorenz can store preferences, cached data, and temporary files there. A separate filesystem or a database would also work for this purpose.
About
Web-based HPC dashboard and more
Resources
License
Stars
Watchers
Forks
Releases
No releases published
Packages 0
No packages published
Languages
- JavaScript 58.0%
- CSS 13.1%
- PHP 9.0%
- Perl 8.5%
- Java 6.4%
- HTML 4.6%
- Other 0.4%