Major Update of Loginom Analytical Platform Functionality of users and packages monitoring was implemented. Python support was added. The existing algorithms were optimized. New handler and visualizer appeared, whereas the existing ones were upgraded.
Many changes and new options were introduced into the new updated version. In our opinion, the most important changes are listed below:
Monitoring of active users and busy packages.
In Loginom 6.4 it became possible to embed the calculations made using the Python programming language code into the workflow. CPython is used as the runtime environment, and the Python component operation in the workflow is identical to the JavaScript component operation. Data transformation to Pandas DataFrame structure and vice versa is supported. The popular libraries used for Python data analysis are available for import: NumPy, Pandas, Scikit-learn, etc.
Use of the Python based scripts
There are no development tools in Loginom that are similar to PyCharm and other specialized products. The main Python use case is to imbed the Python calculations into the existing data analysis workflow.
Taking into consideration the fact that the component executes the Python code on server, there are several restrictions concerning its use:
There are also several safety issues. As opposed to JavaScript, the Python operation is not restricted by the runtime environment and the executed code has full access to the operational system and network protocols. In this context, the administrator must provide the definite possibility to use the Python component on server. The whole code is executed on server with rights of the operational system user under whose id Loginom Server is started.
Activation of the Python execution feature
Python 3.4 and higher and also all packages to be used in the operation process must be installed on server for the Python component operation. For more detailed information possible to contact the Technical Support Service.
Opportunities of the JavaScript component use were significantly expanded, and several errors that could previously occur were corrected.
It became possible to set the output data set structure from the code. Such behaviour could be required if the data structure is not known in advance and it is generated in the node execution process. For example, when analyzing the web service response. The following methods can be used to generate a set of the output columns:
It is possible to set the output fields structure directly from the code
The errors connected with incorrect indexation of rows that can cause the following error while JavaScript code executing: "Row number 0 is out of [0; -1] range", or incorrect displaying of a data set in the preview window were corrected.
When importing from the Data Warehouse, it became possible to display data in the hierarchal (tree-like) structure that can considerably simplify the data import configuration if the warehouse had a complex structure.
Tree-Like Structure when Importing from the Data Warehouse
Inheritance possibility for Import from the Data Warehouse was added. It can be useful if connections to one and the same Data Warehouse with different configuration parameters are used in several places. In wizards of connection to the Data Warehouse on MS SQL and Oracle DBMS it became possible to specify the name of the schema in which metadata and DW data are located.
It also became possible to export to the Data Warehouse. Now Loginom enables to perform the whole cycle of the ETL operations connected with the Data Warehouses.
Configuring export to the Data Warehouse
The component that can significantly simplify analytics, the first stages of work with Data quality. For operation, it is required to set the input and output data sets of fields, and then the component automatically detects full doubles and contradictions in the data set. In this case, the doubled rows are considered to be the rows with exact match of the compared fields.
Duplicates and Contradictions Visualizer
Two important changes that enable to expand possibilities of the Logistic Regression use and in some cases to increase the quality of the received models were introduced into Loginom 6.4.
Setting weights for records while Logistic Regression configuring
Configuration of events to non-events ratio correction
Support of PUT, DELETE and PATCH methods was added in REST request connection wizards. Previously, only POST and GET methods were available. Such methods could be required for operation of some web services.
Previously, http status of the response received from the server was not taken into account, and its value was returned in none of the output ports. The column that contains the web server response code was added in the output data set - "Additional data". The column contains integer values with HttpStatusCode name and "HTTP Status Code" caption.
In some cases, the identifier value of the request received when executing the workflow node by means of Loginom Integrator could be required in the workflow. Previously, it was not possible to get and use this identifier. Now RequestId variable was added to "Workflow Variables" node to "Session variables" port. Its value matches either null value by default, or the request identifier value when executing the workflow node by means of Loginom Integrator.
In the teamwork process or when executing the workflows with complex dependences, it frequently happened that the server notified that the required package was already used. And it was not possible to define the user or process that used the package. Besides, sometimes it was required to clear cache of the packages published by means of Loginom Integrator forcedly, for example, when updating versions of several packages at once.
To tackle these tasks, "Session Manager" that can be used to view the opened sessions and packages and for their monitoring and activity control was added to Administration section. It is also possible to stop the workflow execution forcedly or to reset the user session in this section.
Session Manager Bar of the Administration Section
When debugging occurring errors, sometimes it was required to enable detailed logging of the occuring errors. For this purpose, it was required to restart Loginom Server placing the special file for debugging information record with the executable file. Now it can be done in the server settings and desktop applications. Special parameter "Trace exceptions" was added for this purpose. Server restart is not required in this case but parameter is enabled with a slight delay.
Check of connection string and required parameters was added to the connection wizard. PostgreSQL 12 support was added.
The package export of the strings with the length of 9 and more characters to the fields of BLOB SUB_TYPE TEXT type to the Firebird database was corrected. The following error was corrected: "value" column relates to the money type, and expression - double precision". It occurred while trying to export real numbers to the money field of the PostgreSQL database. The following error was corrected: "value" column relates to the time with time zone type, and expression - timestamp without time zone". It occurred while trying to export the values of the Date/Time type to the time with timezone field of the PostgreSQL database.
It became possible to display several measures simultaneously in the chart graph. Besides, the series shown in the graph can be activated and deactivated.
When filtering by unique values, restriction in displaying not more than 10000 unique items was removed in the Cube. The similar restriction was removed for Row filter. Previously, it was rather inconvenient to set filters by many records, for example, SKU.
The following less significant but still important improvements were also achieved:
Error of zeros removing at the beginning of string fields when exporting from the Cube to Excel was corrected. Frequently it was particularly inconvenient if the fields with item numbers that started from zeros occurred in the exported data sets.
When supplying data to previously trained model, it became possible to compare distribution of actual bin rates with the training ones in Coarse сlasses visualizer. It enables to estimate compliance of the data used when model training and using. If significant inconsistency was detected, it is required to retrain the model.
Comparison of distribution of actual bin rates with the training ones
Variables names and descriptions of rates of events and classes were unified.
To start the node in the Batch processing mode, it is required to set its name, and the node must not be compulsorily published. Previously, it could be done only in the following way: it was required to select the "Published" status in the access modifier configuration window. Then the node name was set and the previous status was returned. Now it became possible to set the name for the model node with the "Public" access modifier. To reset the node name, it is sufficient to leave the field blank (it is possible only for the "Public" access modifier).
The animated icon that shows the status of processes was added in the case of active processes.
Group editing of the data format became possible in the table.
The automated end-to-end testing was added that enabled to enhance the application reliability significantly.
Nodes locking could occur earlier while cutting/inserting the nodes that could cause errors. The locking mechanism became more reliable.
Functionality that enabled to show lag number (offset value) was added for a user to the Correlation Analysis handler when calculating extremum of cross-correlation function.
Names of models parameters were corrected in the Linear and Logistic regressions reports, missing parameters were added to the Linear regression report, names of methods and factors of models selection were added.
It became possible to replace null numerical values with 0 in the Imputation handler, and the handler operation was optimised, Unspecified option was removed from the list of the methods allowable for the Logistic data type.
Tips with full text displayed when hovering a cursor over the field or field caption were added.
The lower bar with the only "Close" button was removed in the quick view window. Instead, the window can be closed by clicking on any space outside the quick view window or by pressing «×» (window close button).
"Scripting" group of components was added (includes Python and JavaScript).
Earlier detected errors were corrected, operational stability of the application was increased, memory consumption was optimised.