From version 5.90a onwards, toolstar®testLX and toolstar®testWIN saving JSON documents in the MongoDB database. This makes it possible to store and archive protocols in a structured and easily queryable manner.
MongoDB database
MongoDB is a schemaless NoSQL database and works with the JSON format (explicitly with a special binary JSON format (BSON) for efficient storage of the data) . Unlike classic SQL databases, which are organized into databases, tables, columns and rows, data in the MongoDB is stored in databases, collections and documents. Unlike SQL databases, each document in a collection can have a different format (different data). This has the advantage that you don't have to build a generic, generally suitable structure before saving the data.
Setting up the database
You can obtain the database installer at https://www.mongodb.com /try/download/community. The database can be easily set up via a guided graphical installation on Windows. The default port for the database after installation is 27017.
After installation, you can log in via MongoDB Compass (should be automatic during installation installed). You can then add a new database using “Create database”. In this newly created database, you then create a collection in which the documents (toolstar® protocols) are then stored. You can do all of this via the graphical interface of MongoDB Compass or via the command line the database.
Facility in the program
In the programs toolstar®testLX and toolstar®testWIN the storage target is set up equivalently within the endurance tests. Basically, you must provide the following information for the reports and the database storage target (all settings assume a standard installation of the MongoDB database):
- Database name: The name of the database you created
- Database name: Den Namen der von Ihnen erstellten Datenbank
- Collection name: The name of the collection you created
- Format: JSON
MongoDB Compass as a graphical interface for the database
In this example there is a database called TestWIN and a collection TestWIN. This includes 3 documents with different information.
With the MongoDB Compass interface you can do the following things:
- Create and manage databases
- Create and manage collections
- View, export and edit the saved data
- Generate and save queries using the interface
- Create and manage indexes
- Make settings for the database and collections
Limitations
Even if the database does not have a schema, it still needs to enforce some rules so that the data can be stored and queried properly. Therefore, not all reports can be saved in full. A very good configuration for storing information would look like this:
In this example, the flow log was explicitly deselected because it contains duplicate data and therefore cannot be saved in MongoDB. Our recommendation is to select a system overview (one-page, detailed, etc.), not a flow log and to evaluate the test results using the hierarchical log instead. You can save the hierarchical log at any level of detail.
Query the data
In the MongoDB Compass interface, you can configure a query using the option switch within the filter field of a collection. Basically, queries are structured like JSON documents and the field names represent a path to a document. In our example we assume the following document in the database:
In this example document from toolstar®testWIN we find some objects ( report, document) and an array (list (information on PC & tester)). The query to find the document with the date 2022-11-04 looks like this:
In this query we find the following settings:
- Filter: {„report.document.datetime“:{$gte:ISODate(‚2022-11-03T00:00:00Z‘).toISOString()}}
- Projection: {„report.document.id“ : 0}
- Sorting: {„_id“ : 1}
- Skip: 0
- Limit: 5
- Max Times MS: 5.000
In most cases you will only need the filter. This specifies which documents should be queried and represents a search question. In this case, the question we asked is: Give me all documents whose field is report.document.datetime (where the points represent a kind of tree structure) whose value greater than equal ($gte) corresponds to a date in ISO format 2022-11-03T00:00:00Z.
The Projection specifies which fields should be retrieved (value: 1) or not retrieved (value: 0). In this example we request all fields except the report.document.id field.
A collation is a set of rules for the order and correspondence of characters that apply to a specific language and locale. In most cases, you can ignore this field and rely on MongoDB's default settings.
Skip and Limit allow you to specify how much data to retrieve. This is especially important if you already have many thousands of documents in the database. The parameters Skip and Limit are responsible for the so-called Paging. This allows you to run through a complete, very large database without overloading your PC. In our example, we retrieve 5 documents in total and skip 0. If we assume that you have a database with 500 entries and want to retrieve them all one after the other, you could set Skip: 0 and Limit: 10 in the first query. In the second query you then set Skip: 10 and Limit: 10 to retrieve documents 10-20.
Tips and ticks
- Efficient queries through indexes
It can make sense to set indexes on certain fields that are frequently accessed so that queries run quickly. A good field for this could be the CPU name if you often call up all devices with Intel or AMD CPUs. - Dashboard for the company
There are many different drivers available for the MongoDB database (Java, PHP, Go, C, C++, etc.). With these drivers you can set up your own applications at any time, which then retrieve the data and display it in an internal dashboard. - Several collections for better sorting
You can of course create many different collections and use them in different reports in toolstar®testLX and toolstar®testWIN. This allows you to sort documents better and make queries easier. You could also regularly move all reports into some kind of archive collection.
