admin/AndroidChart
1
0
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases Wiki Activity

的

Browse Source
This commit is contained in:
coco
2026-07-03 15:56:07 +08:00
commit caef23209c
5767 changed files with 1004268 additions and 0 deletions
Download Patch File Download Diff File Expand all files Collapse all files
binjr/.gitignore
+17
View File
@@ -0,0 +1,17 @@
.idea
*.iml
/binjr/dependency-reduced-pom.xml
/log4j2.xml
binjr.log
launcher/
target/
out/
build/
/.gradle/
/plugins/
/logs/
/.profileconfig.json
binjr/CHANGELOG.md
+721
View File
@@ -0,0 +1,721 @@
## [binjr v3.18.0](https://github.com/binjr/binjr/releases/tag/v3.18.0)
Released on Wed, 24 Apr 2024
_[New]_ Added a new "System" UI theme that inherits the OS color scheme preferences.
_[New]_ Updated embedded OpenJDK and JavaFX runtimes to 22.0.1
_[New]_ Migrated Wix toolset config to version 5.0.0
_[New]_ Added an option to force using the embedded JVM certificate store instead of the host's on Windows and macOS.
_[Fixed]_ Broken parsing profile for unified JVM logs containing only elapsed time.
_[Fixed]_ Notification popup does not show if the text it contains is too long.
## [binjr v3.17.0](https://github.com/binjr/binjr/releases/tag/v3.17.0)
Released on Wed, 21 Mar 2024
_[New]_ Updated the embedded runtimes for Java and JavaFX to version 22.
_[New]_ Support for new package managers: AUR for Archlinux and winget for Windows.
_[New]_ Added an option to override hardware acceleration support.
_[New]_ Added an option to change the user interface scaling factor.
_[New]_ Added an option to trim extraneous spaces in malformed CSV files in parsing profiles.
_[Fixed]_ Dead links for support pages in MSI installer metadata.
## [binjr v3.16.0](https://github.com/binjr/binjr/releases/tag/v3.16.0)
Released on Wed, 7 Feb 2024
_[New]_ Updated the embedded runtimes for Java and JavaFX to version 21.0.2
_[New]_ Added an option to allow Basic auth in tunneling over https.
_[Fixed]_ Source pane remains opened if a connection failed.
_[Fixed]_ Replace Apache http client by OpenJDK's built-in implementation.
_[Fixed]_ Use https instead of http when inferring missing protocol.
## [binjr v3.15.0](https://github.com/binjr/binjr/releases/tag/v3.15.0)
Released on Tue, 5 Dec 2023
_[New]_ Added an option to ignore samples with an undefined Y value instead of forcing them to zero ("Settings > Charts > Treat undefined Y values as 0").
_[New]_ A notification popup now shows download progress when the app is being updated.
_[Fixed]_ `NaN` values produce duplicated samples after Largest-Triangle-Three-Buckets algorithm is applied.
_[Fixed]_ Pagination mechanism when fetching data from index does not honor forceNanToZero property.
_[Fixed]_ A race condition in TimeSeriesProcessor.
_[Fixed]_ Continuously clicking on "Check for update" results in queuing as many download task.
## [binjr v3.14.0](https://github.com/binjr/binjr/releases/tag/v3.14.0)
Released on Mon, 30 Oct 2023
_[New]_ Updated the embedded runtimes for Java and JavaFX to version 21.0.1
_[New]_ Source and target compatibility level for binjr's artifacts have been updated to 21
_[New]_ It is now possible to set the default values for chart type and unit prefixes, used when these aren't defined by the source
_[New]_ Enhanced JVM logging parsing profiles to accept ISO timestamps
_[New]_ Clicking on the find or filter button in a log worksheet now sets focus on the relevant input field
## [binjr v3.13.0](https://github.com/binjr/binjr/releases/tag/v3.13.0)
Released on Wed, 9 Aug 2023
_[New]_ Added new JDK Flight Recorder data adapter.
_[New]_ Users can now choose a worksheet's type when creating a blank one.
_[New]_ Added new type of prefix formatting for charts axis: Percentage.
_[New]_ Added an icon representing the visualization type to sources panes and worksheet tabs.
_[Fixed]_ When there are too many facet pills in an event worksheet, they now overflow to popup menu.
_[Fixed]_ The UI themes in the Settings panel are now listed in alphabetical order.
_[Fixed]_ LogWorksheetController instances cannot be loaded prior to an adapter acquiring Indexes.LOG_FILES
_[Fixed]_ UserInterfaceThemes services cannot be loaded from registered plugin path.
## [binjr v3.12.0](https://github.com/binjr/binjr/releases/tag/v3.12.0)
Released on Sat, 20 May 2023
_[New]_ *binjr* now defaults to an indexing strategy for log files that is optimized for partial terms search and filtering. It allows for fast matching of arbitrary character sequences without the need for explicit syntax like wildcards.
> To revert back to the old behavior that favors searching for whole words, go to "Settings > Logs" and select "Optimize index for whole words search".
_[New]_ Added the ability to open a single log file instead of a whole folder or a zip archive.
_[New]_ Inline help is now directly accessible for many options throughout the application's User Interface by clicking the `?`next to it.
_[New]_ Let users choose the date & time that serves as an anchor to construct timestamps for partial data.
_[New]_ Added an option for the user to toggle whether or not the Y axis should always include the origin (0) when auto-scale is enabled.
_[New]_ Updated embedded runtimes to OpenJDK 20.0.1 and OpenJFX 20.0.1
_[New]_ Added more built-in parsing profiles (Quarkus, Syslog).
_[Fixed]_ Removed unnecessary scoring computation in Log adapter queries to increase filtering performances.
_[Fixed]_ Settings panel is now wider and its content less cramped.
_[Fixed]_ Different outcome when typing the name of a capture group vs selecting it from the dropdown list in the profile editor.
_[Fixed]_ Files selection on Linux do not show files with no extensions when "All files" filter is selected.
_[Fixed]_ Log file adapter does not list files without extensions.
_[Fixed]_ Log file view does not use a monotype font on macOS and Linux.
_[Fixed]_ Auto-update feature ignores alternative signing openPGP signature.
## [binjr v3.11.0](https://github.com/binjr/binjr/releases/tag/v3.11.0)
Released on Wed, 1 Feb 2023
* _[New]_ Added the possibility to parse months from their names (english only for now) in logs and CSV parsing rules.
* _[New]_ Added built-in parsing profile for IcedTea-Web log files.
* _[New]_ Application bundles are now built with, and embed, the Eclipse Temurin distribution of OpenJDK.
* _[New]_ Updated embedded runtimes to OpenJDK 19.0.2 and OpenJFX 19.0.2.1
* _[Fixed]_ Removed unused dependencies to gtk2 package in rpm build.
* _[Fixed]_ Rpm package cannot be built using rpm v4.16.0 or later.
## [binjr v3.10.0](https://github.com/binjr/binjr/releases/tag/v3.10.0)
Released on Thu, 6 Oct 2022
* _[New]_ It is now possible to adjust the size of the text in logs worksheets, as well as changing the default size in the preferences (`Settings > Logs > Default text size`).
* _[New]_ Updated Java and JavaFX Runtimes to version 19.
* _[Fixed]_ Text in debug console could be rendered using wrong encoding on some platforms.
## [binjr v3.9.0](https://github.com/binjr/binjr/releases/tag/v3.9.0)
Released on Thu, 1 Sep 2022
* _[New]_ The csv plugin has been entirely rewritten and now features:
* A new, off heap, backend which allows for working with large quantities of data.
* A much larger selection of user-configurable parameters for parsing CSV files.
* A brand new User Interface to set these parameters and interactively test their effects on sample data.
* The ability to organize, save and import user defined sets of parameters as "profiles" so that they can be reused and shared.
* _[New]_ It is now possible to cancel the loading of log files if it takes too long.
* _[Fixed]_ It is now possible to zoom in on a time interval shorter than one second on charts.
* _[Fixed]_ Resetting the time range on a chart worksheet no longer only takes the first chart into account.
* _[Fixed]_ The reference date for the predefined ranges on the time range picker is now based on the boundaries reported by the adapter.
* _[Fixed]_ Loading indicator causes high GPU usage.
## [binjr v3.8.0](https://github.com/binjr/binjr/releases/tag/v3.8.0)
Released on Wed, 18 May 2022
* _[New]_ It is now possible to reorder worksheet tabs using drag and drop.
* _[New]_ Use a more robust cache mechanism for data fetched from data adapters.
* _[New]_ It is now possible to override the root location for *binjr* temporary folders and files, by setting the `temporaryFilesRoot` property.
## [binjr v3.7.0](https://github.com/binjr/binjr/releases/tag/v3.7.0)
Released on Wed, 23 Mar 2022
* _[New]_ It is now possible to set distinct parsing rules for log files retrieved from a single source, and to swap or edit parsing profiles after a file was added to a worksheet.
* _[New]_ Enhanced snapshot feature to provide a preview of the snapped image and allows users to either save it to a file or to the clipboard.
* _[New]_ Added contextual menu entries for copying series details from tree view and table view.
* _[New]_ *binjr* can now be built to run natively on aarch64 architectures on Linux and macOS.
* _[New]_ Updated Java and JavaFX Runtimes to version 18.
* _[Fixed]_ Pressing the "delete" key while editing series name removes it from worksheet.
* _[Fixed]_ Leaves in source treeview are not sorted in alphabetical order.
* _[Fixed]_ A deadlock can occur if an error is raised while parsing log events.
* _[Fixed]_ It is not possible to browse a folder for log files if some of its children are not accessible to the current user (e.g. due to lack of permission for instance).
*
## [binjr v3.6.0](https://github.com/binjr/binjr/releases/tag/v3.6.0)
Released on Fri, 21 Jan 2022
* _[New]_ Added support for proxy on all HTTP-based data adapters.
* _[New]_ It is now possible to use regular expressions when filtering the source tree view.
* _[New]_ Added a context menu to the list of series in a worksheet to expose edition features (select, delete, rename, etc...)
* _[New]_ Introduced features to automatically infer names and colors for multiple series in a worksheet.
* _[New]_ Updated the embedded OpenJDK and JavaFX runtimes to 17.0.2
* _[Fixed]_ A memory leak cause by a regression introduced in JavaFX 17.0.0
* _[Fixed]_ Improved performances and reduced memory usage when working with logs.
* _[Fixed]_ A regression introduced in v3.3.0 that caused logs with messages already displayed in debug console to not
be displayed there again until console is cleared.
* _[Fixed]_ Selecting a series color using the "custom colors" panel from the color picker does not change the graph.
## [binjr v3.5.2](https://github.com/binjr/binjr/releases/tag/v3.5.2)
Released on Fri, 17 Dec 2021
* _[Fixed]_ Updated Log4j to version 2.16.0 in response to [CVE-2021-45046](https://nvd.nist.gov/vuln/detail/CVE-2021-45046)
## [binjr v3.5.1](https://github.com/binjr/binjr/releases/tag/v3.5.1)
Released on Sat, 11 Dec 2021
* _[Fixed]_ Updated Log4j to version 2.15.0 in response to [CVE-2021-44228](https://nvd.nist.gov/vuln/detail/CVE-2021-44228)
## [binjr v3.5.0](https://github.com/binjr/binjr/releases/tag/v3.5.0)
Released on Thu, 25 Nov 2021
* _[New]_ Added a bar chart to log worksheets that shows the distributions over time of events' severity.
* _[New]_ The precision of the time axis for charts has been increased from seconds to milli-seconds.
* _[New]_ Added the option to skip the confirmation dialog on closing worksheet tabs or charts.
* _[New]_ Added the ability to restore closed tabs using the `Ctrl`+`Shift`+`t` shortcut, similarly to web browsers.
* _[New]_ Enhanced the pagination control on log worksheets with the ability to jump directly to the first, last or arbitrary page.
* _[Fixed]_ After duplicating a log worksheet, changing properties of the log files (in the bottom view) affects both the original and duplicated worksheet.
* _[Fixed]_ Navigating backward or forward on a log worksheet does not change the timeline of linked worksheets.
## [binjr v3.4.0](https://github.com/binjr/binjr/releases/tag/v3.4.0)
Released on Thu, 22 Oct 2021
* _[New]_ Added the ability to zoom and pan on charts using the mouse wheel (or swipe motions on touch devices):
* `Shift + scroll up/down`: Enlarges / reduces the height of all charts in the current worksheet.
* `Ctrl + scroll up/down`: Zooms in / out on the time range for the current worksheet.
* `Alt + scroll up/dowm`: Pans all charts to the left / right in the current worksheet.
* _[New]_ It is now possible to adjust the minimum height of charts in a worksheet.
* _[Fixed]_ Series legend panel in chart worksheets does not fill all available space.
## [binjr v3.3.0](https://github.com/binjr/binjr/releases/tag/v3.3.0)
Released on Thu, 16 Sep 2021
* _[New]_ Updated the embedded runtimes for Java and JavaFX to version 17.
* _[New]_ Source and target compatibility level for binjr's artifacts have been updated to 17.
* _[New]_ Added the ability to search for and highlight keywords in the application's logs in the debug console.
* _[New]_ Changed the format of macOS installable from a `dmg` image to a `pkg` installer.
> The reason behind this change in format for the macOS deliverable is that unsigned DMG images produced by jpackage
> could be reported as "corrupted" under certain conditions instead of just warning users that the application was not
> recognized by the Apple notary service.
> Bundles generated as PKG installers do not exhibit the same problem and correctly warns users that the application is
> not recognized by Apple (and not that the package is corrupted).
* _[Fixed]_ It is not possible to select and copy portions of the logs output in the debug console.
* _[Fixed]_ The button to empty the search text field in log work worksheets is displayed even if the field is already empty.
* _[Fixed]_ Clean-up phase could sometime be skipped on closing worksheets, leading to potential memory leaks.
* _[Fixed]_ Inconsistent content type checking when processing http responses from data adapters.
## [binjr v3.2.0](https://github.com/binjr/binjr/releases/tag/v3.2.0)
Released on Thu, 12 August 2021
* _[New]_ Keep the most recently used log filters in the user history and allow users to save filters as favorites.
* _[Fixed]_ Freeze when clicking on `Browse` to open a file or folder on Linux.
* _[Fixed]_ The entered URL may be ignored when pressing "Enter" in the new source dialog.
* _[Fixed]_ An error occurs when accepting an empty path in the RRD and CSV adapters dialogs.
## [binjr v3.1.0](https://github.com/binjr/binjr/releases/tag/v3.1.0)
Released on Wed, 30 July 2021
* _[New]_ It is now possible to select multiple charts in a worksheet to delete them all at once.
* _[New]_ It is now possible to re-organize the position of charts in a worksheet using drag and drop.
* _[New]_ The panel that shows the series details for charts has been redesigned to make better use of available screen real estate.
* _[New]_ It is now possible to edit the name of a timeseries after it has been added to a worksheet.
## [binjr v3.0.2](https://github.com/binjr/binjr/releases/tag/v3.0.2)
Released on Fri, 30 Apr 2021
* _[Fixed]_ Application fails to start on Windows if Visual Studio 2019 Redistributable is not installed.
## [binjr v3.0.1](https://github.com/binjr/binjr/releases/tag/v3.0.1)
Released on Tue, 6 Apr 2021
* _[Fixed]_ Publication to APT and RPM repositories on new releases is broken.
* _[Fixed]_ Incorrect progression reporting when indexing more than one log file simultaneously.
## [binjr v3.0.0](https://github.com/binjr/binjr/releases/tag/v3.0.0)
Released on Wed, 31 Mar 2021
* _[Breaking Change]_ The plugin API for *binjr* v3.0.0 is no longer compatible with previous versions.
* _[New]_ *binjr* is now able to handle and render time series with data types other than numerical values.
* _[New]_ *binjr* can now extract timeseries data from log files to navigate and filter through log events , in sync with other sources.
* _[New]_ *binjr* can now be run under the Eclipse OpenJ9 JVM
* _[New]_ Relative presets in the time range selection panel.
* _[New]_ Users no longer have to input a minimum of 3 characters in the source filtering bar to trigger filtering.
* _[New]_ Added a new PERF log level in between INFO and DEBUG.
* _[New]_ Embedded Java runtime updated to OpenJDK 16 and OpenJFX 16.
* _[New]_ Added a "Reset Time Range" button to TimeRangePicker control.
* _[New]_ Added new keyboard shortcuts to close a worksheet and navigate history.
* _[New]_ Windows installer allows overriding existing installation path via an MSI property.
* _[New]_ Added the option to display numerical values on charts without unit prefixes.
* _[Change]_ Icons and labels for switching to/from 'Edit' and 'Presentation' mode changed to 'Expand/Reduce Series Views'
* _[Fixed]_ If an error occurs while loading an adapter, all subsequent adapter aren't loaded.
* _[Fixed]_ A sharp performance drop when zooming extremely close up on the time axis (i.e. displaying less than a few seconds)
* _[Fixed]_ Removed unused time zone selection field on Netdata adapter dialog.
* _[Fixed]_ Snapshots taken with the default output scaling use the main monitor scaling factor instead of the one on which the window is displayed.
* _[Fixed]_ Error occurring while fetching data from a single adapter prevents plotting the data recovered from other adapters.
* _[Fixed]_ Modified "New Tab" and "Save As" keyboard shortcuts to be more consistent with well known applications.
* _[Fixed]_ Clicking on an expended source tab's title does not cause it to collapse its contents.
* _[Fixed]_ Pressing `enter` or loosing focus from text entry field when editing source tab title does not validate entry.
* _[Fixed]_ Charts are blurry when binjr is displayed on a screen with a 125%, 150% or 175% scale ratio.
* _[Fixed]_ A concurrent modification exception when applying sampling reduction pre-processing on series.
* _[Fixed]_ Changes to Y axis scale in chart properties are not taken into account by navigation history.
## [binjr v2.17.0](https://github.com/binjr/binjr/releases/tag/v2.17.0)
Released on Thu, 02 Jul 2020
* _[Fixed]_ Jitter on the y-axis when hovering over charts with full height crosshair.
* _[Fixed]_ Incorrect capitalization on some menu entries and labels.
## [binjr v2.16.0](https://github.com/binjr/binjr/releases/tag/v2.16.0)
Released on Wed, 10 Jun 2020
* _[New]_ binjr can now be used as a "portable" application on all supported platforms.
Portable apps can be unpacked to and used from a detachable drive or a file share.
Portable bundles are available in the following formats:
* `tar.gz` for Linux
* `tar.gz` for macOS
* `zip` for Windows
* _[New]_ Alternatively, it can be used as an "installable" application on all supported platforms.
Installable apps integrates with the host OS to provide menu shortcuts, file associations and per user settings.
Installers are available in the following formats:
* `deb` for Debian & Ubuntu
* `rpm` for RHEL, Centos & Fedora
* `dmg` for macOS
* `msi` for Windows
> **IMPORTANT NOTE**: When upgrading an existing copy of the Linux `tar.gz` distribution to version 2.16.0 or later, any previously set preferences will be reset, since it now defaults in "portable" mode and settings are stored directly into the application folder.
You can override this behaviour by adding the command line option ` -Dbinjr.portable=false` when starting the application. You can also use the built-in settings import/export functions to migrate settings from one mode to another.
## [binjr v2.15.0](https://github.com/binjr/binjr/releases/tag/v2.15.0)
Released on Tue, 12 May 2020
* _[New]_ A new adapter allows to use Netdata (https://netdata.cloud) servers as data sources.
* _[New]_ Users can now choose which default color palette to use for charts when the color isn't specified by the source.
* _[New]_ "Show outline" and "Default opacity" preferences are now settable separately for "area charts" and "stacked area" charts.
* _[New]_ Updated the embedded runtime to OpenJDK 14.0.1 and OpenJFX 14.0.1
* _[Fixed]_ JRDS adapter incorrectly reports all charts as stacked area charts.
* _[Fixed]_ "Show outline on area charts " user preference is not persisted across sessions.
* _[Fixed]_ A concurrency issue causes an ArrayIndexOutOfBoundsException when applying sample reduction transform.
* _[Fixed]_ The time range picker is not dismissed automatically after the user selects a preset range.
## [binjr v2.14.0](https://github.com/binjr/binjr/releases/tag/v2.14.0)
Released on Thu, 19 Mar 2020
* _[New]_ Updated the embedded runtime to OpenJDK 14 and OpenJFX 14.
* _[New]_ Linux version no longer depends on GTK 2.
* _[Fixed]_ "Unrecognized image loader:null" error occurs when attempting to capture snapshots of worksheet with many a large number of charts.
## [binjr v2.13.0](https://github.com/binjr/binjr/releases/tag/v2.13.0)
Released on Thu, 30 Jan 2020
* _[New]_ Enhanced downsampling algorithm; this allows a more faithful visual representation of series while still dramatically reducing the number of plotted samples.
* _[New]_ Updated the embedded Java and JavaFX runtimes to 13.0.2
* _[New]_ Changed default value for max heap size to 4GB
* _[Fixed]_ Last and first samples for the selected time range are ignored when rendering data from CSV adapter.
* _[Fixed]_ Time range label on screenshots is incorrect.
* _[Fixed]_ Charts on scaled up screenshots taken in "Presentation" mode are blurry.
* _[Fixed]_ Changed the icon for switching to "Edit" mode to a pen as the previously used cog was confusing.
## [binjr v2.12.0](https://github.com/binjr/binjr/releases/tag/v2.12.0)
Released on Fri, 20 Dec 2019
* _[New]_ Automatically adjusts the time range up when dropping series on an existing worksheets, provided no series where already present.
* _[New]_ The macOS application bundle is now available as a DMG image. This allows for better integration with the menu bar and to register workspace file extention.
* _[Fixed]_ The JRDS data adapter does not check the content type before attempting tp parse an http response payload as JSON.
* _[Fixed]_ The CSV Data Adapter cannot deal with columns having the same name in a single file.
## [binjr v2.11.0](https://github.com/binjr/binjr/releases/tag/v2.11.0)
Released on Thu, 25 Nov 2019
* _[New]_ Updated the embedded Java runtime to OpenJDK 13.0.1
* _[New]_ It is now possible to choose the output scale (i.e. the physical pixel density) for snapshots taken from binjr worksheets.
* _[Fixed]_ On HiDPI screens the tooltip representing a tree node when dragging it to a worksheet is not at the right scale.
## [binjr v2.10.0](https://github.com/binjr/binjr/releases/tag/v2.10.0)
Released on Thu, 24 Oct 2019
* _[New]_ New Adapter API method to center worksheets' time interval to be most relevant with regard to sources
* _[New]_ Application logs are now written to disk by default (in temp directory, 1 file per session, only keeps the last 10 files)
* _[Fixed]_ Concurrent modifications to output console's log queue.
* _[Fixed]_ File or folder chooser dialog does not appear when last opened path is invalid.
## [binjr v2.9.0](https://github.com/binjr/binjr/releases/tag/v2.9.0)
Released on Fri, 27 Sep 2019
* _[New]_ Added options to import and export user preferences, as well as clear opened files history.
* _[New]_ Update bundled OpenJavaFX to version 13.
* _[Fixed]_ NPE in JrdsDataAdapter when the adapter is loaded from saved workspace.
* _[Fixed]_ CsvDataAdapter ignores some configuration keys when loaded from saved workspace.
* _[Fixed]_ Fetching data via an adapter may fail silently.
* _[Fixed]_ Charts do no honor the exact time range specified by the user.
* _[Fixed]_ An offset on the time axis between two or more charts may occurs if the sources for them have different resolutions.
* _[Fixed]_ UI themes defined in external plugins aren't loaded if set as the current theme when binjr is started.
* _[Fixed]_ Unexpected cache miss in http data adapters.
* _[Fixed]_ Disabling a DataAdapter in the settings section doesn't prevent it from being present in "Sources > New Sources..." menu.
* _[Fixed]_ Enabled DataAdapter settings are not persisted in between sessions
## [binjr v2.8.1](https://github.com/binjr/binjr/releases/tag/v2.8.1)
Released on Wed, 04 Sep 2019
* _[Fixed]_ Memory leak: a closed worksheet controller remains reachable if an error notification popup is displayed and user preference _"Discard notification after:"_ is set to _"Never"_.
## [binjr v2.8.0](https://github.com/binjr/binjr/releases/tag/v2.8.0)
Released on Tue, 03 Sep 2019
* _[New]_ Now supports the addition of custom UI themes via external plugins.
* _[New]_ Accepts '.xml' as a valid extension for saved workspaces, in addition to '.bjr'
* _[Fixed]_ A regression introduced in 2.7.0 which prevents access to OS specific certificate stores for SSL validation (Windows / macOS)
## [binjr v2.7.0](https://github.com/binjr/binjr/releases/tag/v2.7.0)
Released on Sun, 18 Aug 2019
* _[New]_ Fetching data for a single chart but from different paths is now done concurrently on multiple threads.
* _[New]_ Added support for adapter that don't need a setup dialog box.
* _[Fixed]_ Worksheet masker pane is dismissed before all charts have been refreshed.
* _[Fixed]_ Concurrent modification of TextFlow control in OutputConsole throws an exception.
* _[Fixed]_ Spurious warnings about cookies invalid expires attributes.
* _[Fixed]_ CsvDecoder should not be re-instantiated each time it is called.
## [binjr v2.6.3](https://github.com/binjr/binjr/releases/tag/v2.6.3)
Released on Wed, 07 Aug 2019
* _[Fixed]_ Prevent update check from proposing to download and install an update on macOS, as in-application installation does not work on this platform at the moment.
* _[Fixed]_ Stop deploying all platform specific resources across all platform application bundles.
## [binjr v2.6.0](https://github.com/binjr/binjr/releases/tag/v2.6.0)
Released on Mon, 05 Aug 2019
* _[New]_ Ability to drag branches with many sub levels from the tree and have them rendered as separate charts on a worksheet.
* _[New]_ Better visual feedback when hovering above a worksheet during a drag and drop operation.
* _[New]_ Ability to select multiple nodes from the source tree to drag onto a worksheet.
* _[New]_ Added a filter functionality to the source tree.
* _[New]_ It is now possible to remove a chart or invoke its property page directly from buttons located on top of the chart's Y axis.
* _[New]_ The position and size of the various resizable panes in the UI are now saved alongside the rest of a workspace, so that its appearance can be fully restored on reload.
* _[New]_ Charts legends pane in edit mode can now be scrolled up and down when many charts are added to a single worksheet.
* _[Fixed]_ Vertical scrollbar on chart view in stacked layout hides part of the graph and causes an horizontal scrollbar to appear.
* _[Fixed]_ The "path" column in the chart legend table doesn't fill up the pane.
## [binjr v2.5.0](https://github.com/binjr/binjr/releases/tag/v2.5.0)
Released on Wed, 06 Jul 2019
* _[New]_ Updates can now be downloaded and applied from within the application.
* _[New]_ binjr now remembers its main window's screen position in-between sessions.
* _[New]_ Added a new "Presentation Mode" that maximize the amount of space dedicated to the visualization of charts by hiding the source pane, chart settings pane and displays the chart legends in a condensed view.
* _[New]_ The snapshot functionality has been enhanced to automatically take a snapshot of the whole charts display area of a worksheet, even if this area requires scrolling when displayed in the application.
* _[New]_ Embedded OpenJDK in application bundle has been updated to version 12.
* _[New]_ Defaults to the new Shenandoah garbage collector with the "compact" heuristics, which allows for a larger maximum heap size while keeping actual memory usage reasonable when a large heap is no longer required.
* _[New]_ Warn end-users when trying to add a large number of series to a single chart at once.
* _[New]_ History of previously opened sources is now accessible via a combo box on the selection dialog (as well as through the existing auto-completion feature).
* _[New]_ Charts vertical axis label are now hilited on mouse-over, to better indicate that they are clickable (clicking on an axis selects the chart as the one currently editable when more than one chart are present on a worksheet).
* _[Fixed]_ Unsightly UI theme application on start-up or when detaching tabs.
* _[Fixed]_ If "Span crosshair over all charts" is true and "auto scale Y axis" is off, then selecting a new time range using the mouse results in incorrectly changing the Y axis scale.
* _[Fixed]_ Selecting a timezone in time picker sometime doesn't register.
* _[Fixed]_ Synchronizing timelines across worksheets is broken.
* _[Fixed]_ UI becomes unresponsive when output console displays a large number of lines (>20000).
* _[Fixed]_ Check for a new versions fails due to Github API rate limit being reached.
## [binjr v2.4.1](https://github.com/binjr/binjr/releases/tag/v2.4.1)
Released on Mon, 08 Apr 2019
* _[Fixed]_ The application becomes unresponsive and crashes with an out-of-memory error if it gets overflown with user requests (e.g. continuous clicks on refresh or back/forward buttons).
* _[Fixed]_ NPE when drag-and-dropping a folded tree.
* _[Fixed]_ Tooltips are not styled according to the selected UI theme.
* _[Fixed]_ Chart properties slide pane should not obscure charting area.
* _[Fixed]_ Stroke width slider is grayed out for line and scatter point charts.
* _[Fixed]_ Suggest popup on data adapter dialog doesn't adapt to longer URLs/paths.
## [binjr v2.4.0](https://github.com/binjr/binjr/releases/tag/v2.4.0)
Released on Fri, 29 Mar 2019
* ___[API Change]___ Removed type parameters from the following classes from the Data Adapter API:
* `DataAdapter<T>` is replaced by `DataAdapter`
* `BaseDataAdapter<T>` is replaced by `BaseDataAdapter`
* `HttpDataAdapter<T, A extends Decoder>` is replaced by `HttpDataAdapter`
* `Decoder<T>` is replaced by `Decoder`
* `TimeSeriesBinding<T>` is replaced by `TimeSeriesBinding`
* `TimeSeriesInfo<T>` is replaced by `TimeSeriesInfo`
* `TimeSeriesProcessor<T>` is replaced by `TimeSeriesProcessor`
* _[New]_ Added a "Settings" button to source panes.
* _[New]_ Added an option to hide the source pane (Command bar menu "Sources > Hide Source Pane" or Ctrl+L)
* _[New]_ Added an option to hide charts legend (Worksheet toolbar "Hide Charts Legends" or Ctrl+B)
* _[New]_ Added an option to span the vertical bar of the selection crosshair over the height of all charts in a
worksheets using a stacked layout (Command bar "Settings > Charts > Hide Source Pane > Span crosshair over all charts")
* _[New]_ Buttons in a worksheet's toolbar will now overflow to a menu pane if there is not enough space to display all
of them all at once.
* _[New]_ A visual indication now identifies the currently selected charts on worksheets when there are more than one.
* _[New]_ Clicking on a chart's title in the graphing area new selects it and expands its legend in the bottom pane.
* _[New]_ Added support for small numbers unit prefix (m = milli, µ = micro, n = nano, etc...) for formatting Y axis values.
* _[New]_ Added confirmation dialog when closing one or several worksheet tabs.
* _[Fixed]_ A memory leak that occurs when adding, moving or changing the type of a chart in an existing worksheet.
* _[Fixed]_ Uncaught exception when entering a negative range for a chart's Y axis causes a worksheet to become.
* _[Fixed]_ Keyboard shortcuts do no work on detached tab windows.
* _[Fixed]_ Dialog boxes are sometime drawn with a null width and height on some Linux/KDE platforms.
## [binjr v2.3.1](https://github.com/binjr/binjr/releases/tag/v2.3.1)
Released on Mon, 11 Feb 2019
* _[Fixed]_ Debug console appender is initialized only when console is displayed for the first time.
* _[Fixed]_ Log level changes when entering or leaving debug console.
* _[Fixed]_ Date formatting does not use the system locale.
* _[Fixed]_ Wrong tooltip for time range picker on worksheet.
## [binjr v2.3.0](https://github.com/binjr/binjr/releases/tag/v2.3.0)
Released on Mon, 28 Jan 2019
* _[Fixed]_ Dropping series onto worksheets in main view fail after a detached tab window was closed.
* _[Fixed]_ Scatter charts are drawn using default colours instead of the colours defined in the source.
* _[Fixed]_ binjr fails to start when double-cliking on the launcher script on Linux or macOS.
* _[Fixed]_ DataAdapter plugins do not get loaded when starting binjr from exec-maven-plugin or Graviton.
* _[Fixed]_ JavaFX crashes on focus loss from dialog on macOS 10.14 Mojave.
* _[Fixed]_ Trying to establish a connection via HTTPS fails with " Received fatal alert: handshake_failure".
* _[Fixed]_ DataAdapter never cleans up its resources if if fails when populating source tree view.
## [binjr v2.2.1](https://github.com/binjr/binjr/releases/tag/v2.2.1)
Released on Fri, 11 Jan 2019
* _[New]_ Enhancements to debug mode console
* _[New]_ Added changelog to distribution
* _[Fixed]_ Disabled the forced sync mechanism for Rrd4J NIO backend.
## [binjr v2.2.0](https://github.com/binjr/binjr/releases/tag/v2.2.0)
Released on Sat, 5 Jan 2019
> **Please note**: Starting with this release, the Maven groupID for all the binjr artifacts changes from `eu.fthevenet` to `eu.binjr`
* _[Change]_ The keyboard shortcut to invoke debug mode changed from CRTL+SHIFT+D to F12.
* _[FIxed]_ The debug output console's log view perpetually grows.
## [binjr v2.1.1](https://github.com/binjr/binjr/releases/tag/v2.1.1)
Released on Fri, 28 Dec 2018
* _[New]_ It is now possible to open binary files and XML dumps created with [RrdTool](https://oss.oetiker.ch/rrdtool/) using the Rrd4j data adapter.
__NB:__ This uses Rrd4j 's built-in conversion facilities in order to import the original file's data into a temporary Rrd4j backend, so be aware that opening a very large rrd file (or a very large number of smaller ones) may be slower than expected, due to the necessary conversion process.
* _[Fixed]_ The tree hierarchy for series bindings created with the Rrd4j adapter is incorrect or incomplete
## [binjr v2.1.0](https://github.com/binjr/binjr/releases/tag/v2.1.0)
Released on Thu, 20 Dec 2018
* _[New]_ Added a new data adapter to directly open and plot the content of rrd db files produced by [RRd4j](https://github.com/rrd4j/rrd4j)
* _[New]_ Added a context menu accessible when right-cliking on the tab that provide shortcuts to various manipulations of the tabs (close, edit, duplicate and detach).
## [binjr v2.0.0](https://github.com/binjr/binjr/releases/tag/v2.0.0)
Released on Mon, 26 Nov 2018
>Starting with version 2.0.0, binjr is built to run on Java 11 and beyond.
>
>___Please note that it does not run on previous version of Java.___
>
> If you require a version that runs on Java 8, you can use the latest releases versioned 1.x.x.
* _[New]_ Built to run on Java 11 and beyond, and use the new standalone distribution of OpenJFX (https://openjfx.io/)
The platform specific packages above contain all required dependencies, including the Java runtime; simply download the one for your OS, unpack it and run "binjr" to start.
* _[New]_ It is now possible to link the time line of two or more independent worksheets (i.e. change the time range on one worksheet also affect all linked worksheets).
* _[New]_ It is now possible to copy/paste a time range from one worksheet to another.
* _[New]_ Removed the modal dialog shown when adding a new worksheet; instead new worksheet are set to editable mode upon creation.
* _[Fixed]_ Clicking the "OK" button on new source dialog has no effect when the source adapter is loaded from a faulty plugin.
## [binjr v1.6.0](https://github.com/binjr/binjr/releases/tag/v1.6.0)
Released on Wed, 31 Oct 2018
* _[New]_ Greatly enhanced time range selection on worksheets.
* _[New]_ Changes to source navigation panel's interface to make it clearer.
* _[New]_ When a source connection is closed, all associated series on worksheets are now removed.
* _[New]_ Many minor tweaks and fixes to UI themes.
* _[Fixed]_ Trailing slash in urls prevent connection to JRDS and other http sources.
* _[Fixed]_ An NPE could occur when closing a source with no worksheet.
## [binjr v1.5.3](https://github.com/binjr/binjr/releases/tag/v1.5.3)
Released on Thu, 11 Oct 2018
* _[Fixed]_ Resources from a DataAdapter are not disposed when a source tab is closed.
* _[Fixed]_ Console output window doesn't always acknowledge appearance changes.
## [binjr v1.5.2](https://github.com/binjr/binjr/releases/tag/v1.5.2)
Released on Fri, 5 Oct 2018
* _[New]_ Report the use of an unsupported version of Java
* _[Fixed]_ Detection of missing JavaFX is broken
* _[Fixed]_ Spurious warning messages because of unset variables.
## [binjr v1.5.1](https://github.com/binjr/binjr/releases/tag/v1.5.1)
Released on Wed, 3 Oct 2018
* _[New]_ User can invoke a console that display log output an d change logging verbosity at runtime.
* _[Fixed]_ File picker dialog box doesn't show if last saved folder is invalid.
## [binjr v1.5.0](https://github.com/binjr/binjr/releases/tag/v1.5.0)
Released on Wed, 19 Sep 2018
* _[New]_ Added a "Dark" UI theme. "Modern" UI theme has been renamed "Light", while "Classic" is unchanged.
* _[New]_ Added the possibility to display debug menu and increase log verbosity at runtime.
* _[Fixed]_ JRDS adapter fails to connect to source if a url contains a trailing slash.
* _[Fixed]_ NPE when initiating a drag & drop motion on an empty tab pane.
* _[Fixed]_ Application cannot start if the UI theme name stored in user preference is not valid.
* _[Fixed]_ The labesl on command bar items sometimes remains visible when the command bar is reduced.
## [binjr v1.4.3](https://github.com/binjr/binjr/releases/tag/v1.4.3)
Released on Mon, 10 Sep 2018
* _[Fixed]_ Built-in DataAdapter are not loaded if an error occurs while scanning the plugin location at startup.
* _[Fixed]_ binjr takes a long time to start because scanning for DataAdapter at visits all sub-folders with maximum depth in plugin location.
* _[Fixed]_ DirectoryChooser dialog doesn't show up if current plugin location if invalid/not a folder
## [binjr v1.4.1](https://github.com/binjr/binjr/releases/tag/v1.4.1)
Released on Tue, 4 Sep 2018
* _[New]_ The duration after which popups automatically fade away can now be configured.
* _[New]_ Relaxed the parsing of URLs when adding a new source (infers a default protocol and port if omitted)
* _[Fixed]_ Failing when a malformed URL is entered for a new JRDS source does not offer a useful error notification.
* _[Fixed]_ Chart background is gray when multiple chart are displayed in stacked view mode but white when overlaid.
## [binjr v1.4.0](https://github.com/binjr/binjr/releases/tag/v1.4.0)
Released on Thu, 2 Aug 2018
* _[New]_ binjr's functionalities can now be extended through the use of plugins.
For the time being, plugins can be used to implement new data source adapters, in order to make binjr capable to communicate with other source systems without the need to change anything to the core module itself.
* _[New]_ The artifact for the core binjr module, which is the sole dependency for building external plugins, is now available via [Maven Central](https://search.maven.org/%23artifactdetails%7Ceu.fthevenet%7Cbinjr%7C1.4.0%7Cjar).
## [binjr v1.3.4](https://github.com/binjr/binjr/releases/tag/v1.3.4)
Released on Wed, 27 Jun 2018
* _[New]_ Performs a sanity check when loading workspaces from files to verify format version number and alert user with a clear error message if it is incompatible.
* _[New]_ Added the option to choose the layout of multiple charts on a single worksheet, either stacked on top of each other, or as an overlay, sharing the same X axis.
* _[Fixed]_ Charts rendering performances greatly improved when visualizing many charts on a single worksheet.
* _[Fixed]_ Deselecting all time series in the main chart in an overlay view would make times series in other chart disappear.
## [binjr v1.3.0](https://github.com/binjr/binjr/releases/tag/v1.3.0)
Released on Mon, 18 Jun 2018
* _[New]_ It is now possible to more than one chart representation to a single worksheet. All charts have independent Y axis, with their own scale and unit, but share the same X axis (which represent time).
Charts on a single worksheet can be of different types (line, area, scatter points, etc...)
The User Interface has been extended to cater for that new core functionality:
- Y axis boundaries are new settable on a per-chart basis (rather than a per- worksheet).
- Time series bindings can now be dragged and dropped onto any existing charts, a new chart or a new worksheet.
- Chart titles, unit names and unit prefixes can now be changed after a worksheet/chart has been created.
- Time series in a chart can be selected/deselected all at once.
* _[Breaking Change]_ The file format used to persist workspaces had to be changed significantly in order to accommodate features introduced in this release and is no longer compatible with the format used in versions prior to 1.3.0.
Note that neither ascending nor descending compatibility is provided; files created in binjr v1.3.0+ cannot be loaded in older versions and files created in older versions cannot be loaded by binjr v1.3.0+.
## [binjr v1.2.3](https://github.com/binjr/binjr/releases/tag/v1.2.3)
Released on Mon, 19 Feb 2018
* _[New]_ Added support for CSV formatted files to be used as data sources.
* _[New]_ Added support for scatter plot charts.
* _[New]_ Added an option to reset all user settings their to default value.
* _[New]_ It is now possible to modify the timezone for a worksheet after is has been created (option in chart settings panel)
* _[Fixed]_ application closing even if "cancel" is selected on save confirmation dialog
* _[Fixed]_ Minor cosmetic fixes and enhancements (Cross-hair no longer appear in front of chart settings panel, message dialogs use vector graphic icons, spelling in messages and logs, etc...)
* _[Fixed]_ Unhandled exceptions when validating inputs in JRDS source dialog box could make the dialog box not acknowledge user clicking the OK button.
* _[Fixed]_ Missing security policies from embedded JRE in Windows native bundle
* _[Fixed]_ Application fail to start with "Could not create jvm" when using Windows native bundle on a machine without a copy of MSVS C++ runtime redistributable installed.
* _[Fixed]_ Underscores in recently open file names menu are sometime removed.
* _[Fixed]_ Many long standing issues with timezone management.
## [binjr v1.1.0](https://github.com/binjr/binjr/releases/tag/v1.1.0)
Released on Fri, 29 Sep 2017
* _[New]_ Worksheet tabs can now be detached from the main window via a simple drag and drop (similar to a web browser).
* _[New]_ Native platform bundles available for Windows (.msi), MacOS (.dmg) and Linux (.rpm and .deb)
These are platform specific install packages that contain a minimal and independent Java Runtime Environment and executable bootstrap, allowing binjr to run as a stand-alone application.
* _[New]_ binjr workspace files can be associated with the application so that binjr is launched on double clicking a .bjr file. This association is automatically performed by the aforementioned native bundles.
* _[New]_ On Windows and MacOS, root CA certificates stored in the OS keystore are used for SSL validation.
* _[Fixed]_ The modal dialog used for user authentication could appear behind the main stage, hence causing the application to appear frozen.
* _[Fixed]_ A possible Null Pointer Exception when using the source/search feature.
* _[Fixed]_ Aligned button background color in Modern theme with Windows 10 standard controls
## [binjr v1.0.15](https://github.com/binjr/binjr/releases/tag/v1.0.15)
Released on Mon, 24 Jul 2017
* _[New]_ Dialog boxes now support UI Theme
* _[Fixed]_ Wrong style applied to button in date picker control
* _[Fixed]_ Changing series visibility doesn't work if chart type is changed.
* _[Fixed]_ Better exception handling in JRDS dataAdapter: error message displayed to end users should be more relevant and helpful in common error scenario.
## [binjr v1.0.13](https://github.com/binjr/binjr/releases/tag/v1.0.13)
Released on Mon, 26 Jun 2017
* _[New]_ It is now possible to change the type of chart used on worksheet after it's been created.
* _[New]_ User can now set the stroke width on line chart and area charts with an outline.
* _[Fixed]_ Line charts ignore colors set in source.
* _[Fixed]_ A slowdown on plotting large series was introduced in release 1.0.12.
## [binjr v1.0.12](https://github.com/binjr/binjr/releases/tag/v1.0.12)
Released on Thu, 22 Jun 2017
* _[New]_ Displays the value of each series for the instant marked by the current position of the vertical marker.
* _[Fixed]_ Series info in table view aren't refreshed properly when time interval changes.
* _[Fixed]_ Removed obsolete parameters from settings panel.
## [binjr v1.0.11](https://github.com/binjr/binjr/releases/tag/v1.0.11)
Released on Tue, 13 Jun 2017
* _[New]_ Using the "Refresh" button now ignores any previously cached data.
* _[Fixed]_ Sorting JRDS treeview by "All filters" or "All tags" is broken
* _[Fixed]_ Application appears to hang when attempting to close it while it is minimized
## [binjr v1.0.10](https://github.com/binjr/binjr/releases/tag/v1.0.10)
Released on Fri, 9 Jun 2017
* _[Fixed]_ Application doesn't provide a clear reason for not starting when JavaFX runtime is not present.
## [binjr v1.0.9](https://github.com/binjr/binjr/releases/tag/v1.0.9)
Released on Wed, 7 Jun 2017
* _[New]_ Search bar to quickly find items in source tree view.
* _[New]_ Better support for JRDS tree view filters.
* _[Fixed]_ The text in "license" and "acknowledgement" panes in about box is blurry.
## [binjr v1.0.8](https://github.com/binjr/binjr/releases/tag/v1.0.8)
Released on Thu, 1 Jun 2017
* _[Fixed]_ Changes to chart appearance settings (outline, area opacity, etc...) are ignored on area charts.
## [binjr v1.0.7](https://github.com/binjr/binjr/releases/tag/v1.0.7)
Released on Wed, 31 May 2017
* _[New]_ JRDS SourceAdapter now supports authenticating through Kerberos
* _[Fixed]_ Dragged tree node would keep following the mouse pointer after being drop onto a worksheet on Linux
* _[Fixed]_ The landing zone for dropping sources onto empty worksheet pane is now much larger
* _[Fixed]_ An invalid cast exception occurs when rendering line charts.
## [binjr v1.0.6](https://github.com/binjr/binjr/releases/tag/v1.0.6)
Released on Tue, 23 May 2017
- *[Fixed]* JVM does not terminates on its own after the main window is closed.
## [binjr v1.0.5](https://github.com/binjr/binjr/releases/tag/v1.0.5)
Released on Tue, 23 May 2017
- *[New]* Long running tasks, such as loading a workpace or fetching time-series data from sources, are now executed asynchronously to the UI refresh. This increases the global responsiveness of the application and prevents most occurrences of the applications "freezing" for a few seconds during those tasks.
- *[New]* Errors when connecting to a source or parsing a workspace file are now reported as modeless notification popups rather than modal dialog boxes.
- *[New]* The behaviour of the auto-ranging feature for the Y axis has changed; it is now a toggle button, rather than a push button that would reset the range.
- *[Fixed]* An bug in DecimationTransform, causes a "java.lang.IllegalArgumentException: Duplicate data added" exceptions.
## [binjr v1.0.4](https://github.com/binjr/binjr/releases/tag/v1.0.4)
Released on Thu, 18 May 2017
- *[New]* Use drag and drop to add series sources to the current or a new worksheet.
## [binjr v1.0.3](https://github.com/binjr/binjr/releases/tag/v1.0.3)
Released on Wed, 17 May 2017
- *[Fixed]* Pressing 'del' to remove a series from a worksheet also removed all subsequent series in table view.
## [binjr v1.0.2](https://github.com/binjr/binjr/releases/tag/v1.0.2)
Released on Tue, 16 May 2017
- *[New]* Greatly enhanced responsiveness when working with series with large number of samples.
- *[New]* Reworked the UI to display settings and preferences via sliding panes rather than dialog boxes.
- *[New]* The crosshair visibility behaviour has been modified: the vertical marker is now on by default and switching both markers on or off is now remembered across sessions.
- *[Fixed]* Automatic check for updates now limited to once per hour.
- *[Fixed]* NPE in workspace source list listener.
## [binjr v1.0.1](https://github.com/binjr/binjr/releases/tag/v1.0.1)
Released on Tue, 25 Apr 2017
- *[New]* Added a feature to automatically check for new releases.
- *[Fixed]* An empty tree view is displayed when after attempt to add a source failed.
- *[Fixed]* Connecting an to invalid source fails silently.
- *[Fixed]* The application hangs while manipulating the tree view when running under Windows 10.
## [binjr v1.0.0](https://github.com/binjr/binjr/releases/tag/v1.0.0)
Released on Fri, 14 Apr 2017
Initial release
binjr/CONTRIBUTING.md
+11
View File
@@ -0,0 +1,11 @@
# How to Contribute
The binjr project is [Apache 2.0 licensed](LICENSE.md) and accepts contributions via
GitHub pull requests.
## Certificate of Origin
By contributing to this project you agree to the Developer Certificate of
Origin (DCO). This document was created by the Linux Kernel community and is a
simple statement that you, as a contributor, have the legal right to make the
contribution. See the [DCO](DCO.md) file for details.
binjr/DCO
+37
View File
@@ -0,0 +1,37 @@
Developer Certificate of Origin
Version 1.1
Copyright (C) 2004, 2006 The Linux Foundation and its contributors.
1 Letterman Drive
Suite D4700
San Francisco, CA, 94129
Everyone is permitted to copy and distribute verbatim copies of this
license document, but changing it is not allowed.
Developer's Certificate of Origin 1.1
By making a contribution to this project, I certify that:
(a) The contribution was created in whole or in part by me and I
have the right to submit it under the open source license
indicated in the file; or
(b) The contribution is based upon previous work that, to the best
of my knowledge, is covered under an appropriate open source
license and I have the right under that license to submit that
work with modifications, whether created in whole or in part
by me, under the same open source license (unless I am
permitted to submit under a different license), as indicated
in the file; or
(c) The contribution was provided directly to me by some other
person who certified (a), (b) or (c) and I have not modified
it.
(d) I understand and agree that this project and the contribution
are public and that a record of the contribution (including all
personal information I submit with it, including my sign-off) is
maintained indefinitely and may be redistributed consistent with
this project or the open source license(s) involved.
binjr/LICENSE.md
+201
View File
@@ -0,0 +1,201 @@
Apache License
Version 2.0, January 2004
http://www.apache.org/licenses/
TERMS AND CONDITIONS FOR USE, REPRODUCTION, AND DISTRIBUTION
1. Definitions.
"License" shall mean the terms and conditions for use, reproduction,
and distribution as defined by Sections 1 through 9 of this document.
"Licensor" shall mean the copyright owner or entity authorized by
the copyright owner that is granting the License.
"Legal Entity" shall mean the union of the acting entity and all
other entities that control, are controlled by, or are under common
control with that entity. For the purposes of this definition,
"control" means (i) the power, direct or indirect, to cause the
direction or management of such entity, whether by contract or
otherwise, or (ii) ownership of fifty percent (50%) or more of the
outstanding shares, or (iii) beneficial ownership of such entity.
"You" (or "Your") shall mean an individual or Legal Entity
exercising permissions granted by this License.
"Source" form shall mean the preferred form for making modifications,
including but not limited to software source code, documentation
source, and configuration files.
"Object" form shall mean any form resulting from mechanical
transformation or translation of a Source form, including but
not limited to compiled object code, generated documentation,
and conversions to other media types.
"Work" shall mean the work of authorship, whether in Source or
Object form, made available under the License, as indicated by a
copyright notice that is included in or attached to the work
(an example is provided in the Appendix below).
"Derivative Works" shall mean any work, whether in Source or Object
form, that is based on (or derived from) the Work and for which the
editorial revisions, annotations, elaborations, or other modifications
represent, as a whole, an original work of authorship. For the purposes
of this License, Derivative Works shall not include works that remain
separable from, or merely link (or bind by name) to the interfaces of,
the Work and Derivative Works thereof.
"Contribution" shall mean any work of authorship, including
the original version of the Work and any modifications or additions
to that Work or Derivative Works thereof, that is intentionally
submitted to Licensor for inclusion in the Work by the copyright owner
or by an individual or Legal Entity authorized to submit on behalf of
the copyright owner. For the purposes of this definition, "submitted"
means any form of electronic, verbal, or written communication sent
to the Licensor or its representatives, including but not limited to
communication on electronic mailing lists, source code control systems,
and issue tracking systems that are managed by, or on behalf of, the
Licensor for the purpose of discussing and improving the Work, but
excluding communication that is conspicuously marked or otherwise
designated in writing by the copyright owner as "Not a Contribution."
"Contributor" shall mean Licensor and any individual or Legal Entity
on behalf of whom a Contribution has been received by Licensor and
subsequently incorporated within the Work.
2. Grant of Copyright License. Subject to the terms and conditions of
this License, each Contributor hereby grants to You a perpetual,
worldwide, non-exclusive, no-charge, royalty-free, irrevocable
copyright license to reproduce, prepare Derivative Works of,
publicly display, publicly perform, sublicense, and distribute the
Work and such Derivative Works in Source or Object form.
3. Grant of Patent License. Subject to the terms and conditions of
this License, each Contributor hereby grants to You a perpetual,
worldwide, non-exclusive, no-charge, royalty-free, irrevocable
(except as stated in this section) patent license to make, have made,
use, offer to sell, sell, import, and otherwise transfer the Work,
where such license applies only to those patent claims licensable
by such Contributor that are necessarily infringed by their
Contribution(s) alone or by combination of their Contribution(s)
with the Work to which such Contribution(s) was submitted. If You
institute patent litigation against any entity (including a
cross-claim or counterclaim in a lawsuit) alleging that the Work
or a Contribution incorporated within the Work constitutes direct
or contributory patent infringement, then any patent licenses
granted to You under this License for that Work shall terminate
as of the date such litigation is filed.
4. Redistribution. You may reproduce and distribute copies of the
Work or Derivative Works thereof in any medium, with or without
modifications, and in Source or Object form, provided that You
meet the following conditions:
(a) You must give any other recipients of the Work or
Derivative Works a copy of this License; and
(b) You must cause any modified files to carry prominent notices
stating that You changed the files; and
(c) You must retain, in the Source form of any Derivative Works
that You distribute, all copyright, patent, trademark, and
attribution notices from the Source form of the Work,
excluding those notices that do not pertain to any part of
the Derivative Works; and
(d) If the Work includes a "NOTICE" text file as part of its
distribution, then any Derivative Works that You distribute must
include a readable copy of the attribution notices contained
within such NOTICE file, excluding those notices that do not
pertain to any part of the Derivative Works, in at least one
of the following places: within a NOTICE text file distributed
as part of the Derivative Works; within the Source form or
documentation, if provided along with the Derivative Works; or,
within a display generated by the Derivative Works, if and
wherever such third-party notices normally appear. The contents
of the NOTICE file are for informational purposes only and
do not modify the License. You may add Your own attribution
notices within Derivative Works that You distribute, alongside
or as an addendum to the NOTICE text from the Work, provided
that such additional attribution notices cannot be construed
as modifying the License.
You may add Your own copyright statement to Your modifications and
may provide additional or different license terms and conditions
for use, reproduction, or distribution of Your modifications, or
for any such Derivative Works as a whole, provided Your use,
reproduction, and distribution of the Work otherwise complies with
the conditions stated in this License.
5. Submission of Contributions. Unless You explicitly state otherwise,
any Contribution intentionally submitted for inclusion in the Work
by You to the Licensor shall be under the terms and conditions of
this License, without any additional terms or conditions.
Notwithstanding the above, nothing herein shall supersede or modify
the terms of any separate license agreement you may have executed
with Licensor regarding such Contributions.
6. Trademarks. This License does not grant permission to use the trade
names, trademarks, service marks, or product names of the Licensor,
except as required for reasonable and customary use in describing the
origin of the Work and reproducing the content of the NOTICE file.
7. Disclaimer of Warranty. Unless required by applicable law or
agreed to in writing, Licensor provides the Work (and each
Contributor provides its Contributions) on an "AS IS" BASIS,
WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or
implied, including, without limitation, any warranties or conditions
of TITLE, NON-INFRINGEMENT, MERCHANTABILITY, or FITNESS FOR A
PARTICULAR PURPOSE. You are solely responsible for determining the
appropriateness of using or redistributing the Work and assume any
risks associated with Your exercise of permissions under this License.
8. Limitation of Liability. In no event and under no legal theory,
whether in tort (including negligence), contract, or otherwise,
unless required by applicable law (such as deliberate and grossly
negligent acts) or agreed to in writing, shall any Contributor be
liable to You for damages, including any direct, indirect, special,
incidental, or consequential damages of any character arising as a
result of this License or out of the use or inability to use the
Work (including but not limited to damages for loss of goodwill,
work stoppage, computer failure or malfunction, or any and all
other commercial damages or losses), even if such Contributor
has been advised of the possibility of such damages.
9. Accepting Warranty or Additional Liability. While redistributing
the Work or Derivative Works thereof, You may choose to offer,
and charge a fee for, acceptance of support, warranty, indemnity,
or other liability obligations and/or rights consistent with this
License. However, in accepting such obligations, You may act only
on Your own behalf and on Your sole responsibility, not on behalf
of any other Contributor, and only if You agree to indemnify,
defend, and hold each Contributor harmless for any liability
incurred by, or claims asserted against, such Contributor by reason
of your accepting any such warranty or additional liability.
END OF TERMS AND CONDITIONS
APPENDIX: How to apply the Apache License to your work.
To apply the Apache License to your work, attach the following
boilerplate notice, with the fields enclosed by brackets "{}"
replaced with your own identifying information. (Don't include
the brackets!) The text should be enclosed in the appropriate
comment syntax for the file format. We also recommend that a
file or class name and description of purpose be included on the
same "printed page" as the copyright notice for easier
identification within third-party archives.
Copyright {yyyy} {name of copyright owner}
Licensed under the Apache License, Version 2.0 (the "License");
you may not use this file except in compliance with the License.
You may obtain a copy of the License at
http://www.apache.org/licenses/LICENSE-2.0
Unless required by applicable law or agreed to in writing, software
distributed under the License is distributed on an "AS IS" BASIS,
WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
See the License for the specific language governing permissions and
limitations under the License.
binjr/NOTICE.md
+96
View File
@@ -0,0 +1,96 @@
# binjr
###### A Time Series Data Browser
---
Copyright 2016-2024 Frederic Thevenet
Licensed under the Apache License, Version 2.0 (the "License");
You may obtain a copy of the License at
http://www.apache.org/licenses/LICENSE-2.0
Unless required by applicable law or agreed to in writing, software
distributed under the License is distributed on an "AS IS" BASIS,
WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
See the License for the specific language governing permissions and
limitations under the License.
All source code for the project is available in the following source code repository:
* [https://github.com/binjr/binjr](https://github.com/binjr/binjr)
More info at:
* [https://binjr.eu](https://binjr.eu)
## Third-party
This software uses material from several third party open source software projects, listed below.
A copy of the licenses used by these projects is distributed as part of this software in `resources/licences/`
**Apache Commons CSV**
Copyright 2005-2017 The Apache Software Foundation
Licensed under the [Apache License, Version 2.0](http://www.apache.org/licenses/LICENSE-2.0)
**Apache HttpComponents Client**
Copyright 1999-2017 The Apache Software Foundation
Licensed under the [Apache License, Version 2.0](http://www.apache.org/licenses/LICENSE-2.0)
**Apache Log4j**
Copyright 1999-2017 Apache Software Foundation
Licensed under the [Apache License, Version 2.0](http://www.apache.org/licenses/LICENSE-2.0)
**Apache Lucene**
Copyright 2001-2020 The Apache Software Foundation
Licensed under the [Apache License, Version 2.0](http://www.apache.org/licenses/LICENSE-2.0)
**Bouncy Castle Cryptography APIs**
Copyright (c) 2000 - 2020 The Legion of the Bouncy Castle Inc.
Licensed under the [MIT License](https://opensource.org/licenses/MIT).
**ControlsFX**
Copyright (c) 2013, 2014, ControlsFX. All rights reserved.
Licensed under the [3-Clause BSD License](https://opensource.org/licenses/BSD-3-Clause).
**DateAxis**
Copyright (c) 2013, Christian Schudt.
Licensed under the [MIT License](https://opensource.org/licenses/MIT).
**Eclipse Implementation of JAXB**
Licensed under the [Eclipse Distribution License v. 1.0](https://www.eclipse.org/org/documents/edl-v10.html).
**e(fx)clipse**
Licensed under the [Eclipse Distribution License v. 1.0](https://www.eclipse.org/org/documents/edl-v10.html).
**Fira Code Font**
Copyright (c) 2014, The Fira Code Project Authors (https://github.com/tonsky/FiraCode)
This Font Software is licensed under the SIL Open Font License, Version 1.1 (http://scripts.sil.org/OFL)
**Font Awesome Free**
by @fontawesome - https://fontawesome.com
License - https://fontawesome.com/license/free (Icons: CC BY 4.0, Fonts: SIL OFL 1.1, Code: MIT License)
**Google Gson**
Copyright 2008 Google Inc
Licensed under the [Apache License, Version 2.0](http://www.apache.org/licenses/LICENSE-2.0)
**JAXB adapters for Java 8 Date and Time API types**
Copyright (c) 2015 Mikhail Sokolov
Licensed under the [Apache License, Version 2.0](http://www.apache.org/licenses/LICENSE-2.0)
**jfxutils**
Copyright 2013 Jason Winnebeck
Licensed under the [Apache License, Version 2.0](http://www.apache.org/licenses/LICENSE-2.0)
**Lato Fonts**
Copyright (c) 2010-2014 by tyPoland Lukasz Dziedzic (team@latofonts.com) with Reserved Font Name "Lato"
This Font Software is licensed under the SIL Open Font License, Version 1.1 (http://scripts.sil.org/OFL)
**RichTextFX**
Copyright (c) 2013-2017, Tomas Mikula and contributors
Licensed under the [The 2-Clause BSD License](https://opensource.org/licenses/BSD-2-Clause)
**RRD4J**
Copyright (c) 2011-2019 The RRD4J Authors
Licensed under the [Apache License, Version 2.0](http://www.apache.org/licenses/LICENSE-2.0)
binjr/README.md
+182
View File
@@ -0,0 +1,182 @@
# [![binjr](https://binjr.eu/assets/images/binjr_readme_title.png)](https://binjr.eu)
[![Build Status](https://dev.azure.com/binjr/binjr/_apis/build/status/binjr.binjr)](https://dev.azure.com/binjr/binjr/_build/latest?definitionId=1) [![Github Release](https://img.shields.io/github/release/binjr/binjr.svg?label=Github%20Release)](https://github.com/binjr/binjr/releases/latest) [![Maven Central](https://img.shields.io/maven-central/v/eu.binjr/binjr-core.svg?label=Maven%20Central)](https://search.maven.org/search?q=g:%22eu.binjr%22)
[![trailer](https://binjr.eu/assets/images/binjr_landing_poster.png)](https://binjr.eu/trailer.html)
## Contents
* [What is binjr?](#what-is-binjr)
* [Features](#features)
* [Getting started](#getting-started)
* [Trying it out](#trying-it-out)
* [Getting help](#getting-help)
* [Contributing](#contributing)
* [How is it licensed?](#how-is-it-licensed)
## What is binjr?
_binjr_ is a time series browser; it renders time series data produced by other applications as
dynamically editable representations and provides advanced features to navigate the data smoothly and efficiently
(drag & drop, zoom, history, detachable tabs, advanced time-range picker).
It is a standalone client application, that runs independently of the applications that produce the data; there are
no server or server-side components dedicated to _binjr_ that need to be installed on the source.
The user experience in _binjr_ revolves around enabling users to compose a custom view by using any of the time-series
exposed by the source, simply by dragging and dropping them on the view.
That view then constantly evolves as users add or remove series from different sources, while navigating through it by changing the time range,
the type of chart visualization and smaller aspects such as the color or transparency for each series.
Users can then save the current state of their session at any time to a file, in order to reopen it later or to share it with someone else.
_binjr_ also possesses the ability to visualize time series not only as charts of numeric values, but can be customized to
support visualization for any data type; for instance it features out-of-the-box a source adapter for text based log files.
Log files, produced by applications to trace their lifecycle at runtime, typically contain timestamps for each event
they contain; so we can think of them as time series, but with data points being textual information instead of numerical
values.
In practical terms, this means that a lot of the features built into binjr to compose and navigate time series
visualizations can be applied to log files with great benefits.
Behind the scene, _binjr_ uses Apache Lucene to index data from log files; meaning users can use its powerful query
language to hack through vast quantities logged events.
It also allows _binjr_ to open log files of any size; unlike most text editors which will fail to load multi
gigabytes-sized files as they try to fit it all in memory, _binjr_ will happily index those and present a paginated view so
that memory usage remains reasonable, while the backing index ensures navigating and searching is lightning fast.
With these abilities, _binjr_ aims to become the missing link between text editors and command line tools
traditionally used to analyse monitoring data locally and full-blown log analytics platforms (e.g. Elastic/Logstash/Kibana
stack) that centralizes logs for entire organizations.
It provides many of the same powerful visualization and search features while still remaining a totally
local solution (the data never needs to be pushed to the cloud - or anywhere else for that matter), and requiring no
setup nor maintenance to speak of.
### ...and what it isn't
* _binjr_ is **not** a system performance collector, nor a collector of anything else for that matter. What it provides is
efficient navigation and pretty presentation for time series collected elsewhere.
* _binjr_ is **not** a cloud solution. It's not even a server based solution; it's entirely a client application,
albeit one that can get its data from remote servers. Think of it as a browser, only just for time series.
* _binjr_ is **not** a live system monitoring dashboard. While you can use it to connect to live sources, its feature set is
not geared toward that particular task, and there are better tools for that out there. Instead, it aims to be an
investigation tool, for when you don't necessarily know what you're looking for beforehand and you'll want to build
and change the view of the data as you navigate through it rather than be constrained by pre-determined dashboards.
## Features
#### Data source agnostic
* Standalone, client-side application.
* Can connect to any number of sources, of different types, at the same time.
* Communicates though the APIs exposed by the source.
* Supports for data sources is extensible via plugins.
* Supports time-series with numeric (e.g. charts) or text (e.g. logs) values.
#### Designed for ad-hoc view composition
* Drag and drop series from any sources directly on the chart view.
* Mix series from different sources on the same view.
* Allows charts overlay: create charts with several Y axis and a shared time line.
* Highly customizable views; choose chart types, change series colours, transparency, legends, etc...
* Save you work session to a file at any time, to be reopened later or shared with someone else.
#### Smooth navigation
* Mouse driven zoom of both X and Y axis.
* Drag and drop composition.
* Browser-like, forward & backward navigation of zoom history.
* Advanced time-range selection widget.
* The tabs holding the chart views can be detached into separate windows.
* Charts from different tabs/windows can be synchronized to a common time line.
#### Fast, responsive & aesthetically pleasing visuals
* Built on top of [JavaFX](https://openjfx.io/) for a modern look and cross-platform, hardware accelerated graphics.
* Three different UI themes, to better integrate with host OS and fit user preferences.
#### Java based application
* Cross-platform: works great on Linux, macOS and Windows desktops!
* Strong performances, even under heavy load (dozens of charts with dozens of series and thousands of samples).
## Supported data sources
***binjr*** can consume time series data provided by the following data sources:
| Name | Description | Built-in[1] | Source type |
|-------------------------------------------------------------|--------------------------------------------------------------------------------------------------------------------------------|-------------|-------------|
| [JRDS](https://github.com/fbacchella/jrds) | A performance monitoring application written in Java. | ✓ | Remote |
| [Netdata](https://www.netdata.cloud) | Distributed, real-time performance and health monitoring for systems and applications. | ✓ | Remote |
| RRD Files | Round-Robin Database files produced by [RRDtool](https://oss.oetiker.ch/rrdtool/) and [RRD4J](https://github.com/rrd4j/rrd4j). | ✓ | Local files |
| CSV Files | Comma Separated Values files. | ✓ | Local files |
| Log Files | Text based, semi-structured log files. | ✓ | Local files |
| [JDK Flight Recoder](https://openjdk.org/jeps/328) | Low-overhead data collection framework for troubleshooting Java applications and the HotSpot JVM. | ✓ | Local files |
| [Demo Adapter](https://github.com/binjr/binjr-adapter-demo) | A plugin for binjr that provides data sources for demonstration purposes. | | Local files |
[1]: Support for data sources not marked as *'Built-in'* requires additional plugins.
## Getting started
There are several ways to get up and running with ***binjr***:
#### Download an application bundle
The simplest way to start using ***binjr*** is to download an application bundle from the [download page](https://binjr.eu/download/latest_release/).
These bundles contain all the dependencies required to run the app, including a copy of the Java runtime specially
crafted to only include the required components and save disk space.
They are only ~75 MiB in size and there is one for each of the supported platform: Windows, Linux and macOS.
Simply download the one for your system, unpack it and run `binjr` to start!
#### Build from source
You can also build or run the application from the source code using the included Gradle wrapper.
Simply clone the [repo from Github](https://github.com/binjr/binjr/) and run:
* `./gradlew build` to build the JAR for the all the modules.
* `./gradlew run` to build and start the application straight away.
* `./gradlew clean packageDistribution` to build an application bundle for the platform on which you ran the build.
> Please note that it is mandatory to run the `clean` task in between two executions of the `packageDistribution` in
> the same environment.
## Trying it out
If you'd like to experience binjr's visualization capabilities but do not have a compatible data source handy, you can use
the [demonstration data adapter](https://github.com/binjr/binjr-adapter-demo).
It is a plugin which embeds a small, stand-alone data source that you can readily browse using ***binjr***.
1. Make sure ***binjr*** is installed on your system and make a note of the folder it is installed in.
2. Download the `binjr-adapter-demo-1.x.x.zip` archive from https://github.com/binjr/binjr-adapter-demo/releases/latest
3. Copy the `binjr-adapter-demo-1.x.x.jar` file contained in the zip file into the `plugins` folder of your
***binjr*** installation.
4. Start ***binjr*** (or restart it if it was runnning when you copied the plugin) and open the `demo.bjr`
workspace contained in the zip (from the command menu, select `Workspaces > Open...`, or press Ctrl+O)
## Getting help
The documentation can be found [here](https://binjr.eu/documentation/getting-started/).
If you encounter an issue, or would like to suggest an enhancement or a new feature, you may do so [here](https://github.com/binjr/binjr/issues).
## How is it licensed?
***binjr*** is released under the [Apache License version 2.0](https://github.com/binjr/binjr/blob/master/LICENSE.md).
## Contributing
This project accepts contributions via [GitHub pull requests](https://github.com/binjr/binjr/pulls).
### Certificate of Origin
By contributing to this project you agree to the Developer Certificate of
Origin (DCO). This document was created by the Linux Kernel community and is a
simple statement that you, as a contributor, have the legal right to make the
contribution. See the [DCO](DCO) file for details.
binjr/SECURITY.md
+17
View File
@@ -0,0 +1,17 @@
# Security Policy
This software is provided under the [Apache Software License version 2.0](http://www.apache.org/licenses/LICENSE-2.0).
Unless required by applicable law or agreed to in writing, software distributed under the License is distributed
on an "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. See the License for
the specific language governing permissions and limitations under the License.
Within the limits stated above, we do endeavor to fix bugs and security vulnerabilities that are brought to our
attention, and we truly appreciate the help of the community in trying to improve the quality and safety of this software.
Please assume that fixes for bugs or vulnerabilities will only be made available in the most recent release of the software.
## Reporting a Vulnerability
You can report a security vulnerability privately via this page: https://github.com/binjr/binjr/security
binjr/UNRELEASED.md
View File
binjr/azure-pipelines.yml
+248
View File
@@ -0,0 +1,248 @@
trigger:
branches:
include:
- master
- '*-dev'
- '*-ci_test'
tags:
include:
- v*
name: $(Date:yyyyMMdd)$(Rev:.r)
variables:
system.debug: false
isTag: 'false'
isSnapshot: 'false'
javaVersion: '21'
stages:
- stage: build
jobs:
# Linux app bundle job
- template: build-job.yml
parameters:
name: app_bundle_linux
platform: linux
arch: x64
jdkDownloadUrl: https://api.adoptium.net/v3/binary/latest/$(javaVersion)/ga/linux/x64/jdk/hotspot/normal/eclipse
jdkFile: $(Agent.TempDirectory)/jdk-latest-linux_x64.tar.gz
javaVersion: $(javaVersion)
pool:
vmImage: 'ubuntu-22.04'
# MacOS app bundle job
- template: build-job.yml
parameters:
name: app_bundle_mac
platform: mac
arch: x64
jdkDownloadUrl: https://api.adoptium.net/v3/installer/latest/$(javaVersion)/ga/mac/x64/jdk/hotspot/normal/eclipse
jdkFile: $(Agent.TempDirectory)/jdk-latest-macosx_x64.pkg
javaVersion: $(javaVersion)
pool:
vmImage: 'macOS-12'
# Windows app bundle job
- template: build-job.yml
parameters:
name: app_bundle_windows
platform: win
arch: x64
wixVersion: '5.0.0'
jdkDownloadUrl: https://api.adoptium.net/v3/binary/latest/$(javaVersion)/ga/windows/x64/jdk/hotspot/normal/eclipse
jdkFile: $(Agent.TempDirectory)/jdk-latest-win_x64.zip
javaVersion: $(javaVersion)
pool:
vmImage: 'windows-2022'
# Finalize release
- stage: deploy_site
dependsOn: build
condition: and(succeeded(), startsWith(variables['Build.SourceBranch'], 'refs/tags/'), not(endsWith(variables['Build.SourceBranch'], '-SNAPSHOT')), not(contains(variables['Build.SourceBranch'], '-b')))
jobs:
- job: deploy_site
pool:
vmImage: 'ubuntu-22.04'
steps:
- script: |
curl --output $(Agent.TempDirectory)/jdk-latest-linux_x64.tar.gz -O -J -L https://api.adoptium.net/v3/binary/latest/$(javaVersion)/ga/linux/x64/jdk/hotspot/normal/eclipse
- task: JavaToolInstaller@0
inputs:
versionSpec: $(javaVersion)
jdkArchitectureOption: x64
jdkSourceOption: LocalDirectory
jdkFile: $(Agent.TempDirectory)/jdk-latest-linux_x64.tar.gz
jdkDestinationDirectory: $(Agent.ToolsDirectory)/binaries/openjdk
cleanDestinationDirectory: true
- task: DownloadSecureFile@1
name: gpgKeyring
displayName: 'Download GPG Keyring'
inputs:
secureFile: 'keyring.gpg'
- task: Gradle@2
env:
IS_TAG: 'true'
REPO_TAG_NAME: $(Build.SourceBranchName)
GPG_KEY_NAME: $(gpg.keyname)
GPG_PASSPHRASE: $(gpg.passphrase)
GPG_KEYRING_PATH: $(gpgKeyring.secureFilePath)
OSSRH_TOKEN_PASSWORD: $(ossrh.token.password)
OSSRH_TOKEN_USERNAME: $(ossrh.token.username)
inputs:
gradleWrapperFile: 'gradlew'
javaHomeOption: 'JDKVersion'
jdkVersionOption: 1.$(javaVersion)
jdkArchitectureOption: 'x64'
publishJUnitResults: false
tasks: 'expandMdTemplates'
- bash: |
python3 --version
python3 -m pip --version
python3 -m pip install --upgrade pip setuptools
python3 -m pip install mkdocs
python3 -m pip install mkdocs-material==9.*
git clone -b sources https://github.com/binjr/binjr.github.io build/tmp/binjr-site
cd build/tmp/binjr-site
cp ../expanded/CHANGELOG.md docs/download/CHANGELOG.md
cp ../expanded/latest_release.md docs/download/latest_release.md
echo "https://binjr-bot:$GH_ACCESS_TOKEN@github.com" >> "$HOME/.git-credentials"
git config credential.helper store
git config user.email "binjr.bot@free.fr"
git config user.name "binjr-bot"
wget https://api.github.com/repos/binjr/binjr/releases/latest -O docs/repos/binjr/binjr/releases/latest
git commit -am "Release $BUILD_SOURCEBRANCHNAME"
git push
python3 -m mkdocs gh-deploy --no-history --remote-branch master
git config credential.helper cache
shred -fuz "$HOME/.git-credentials"
env:
GH_ACCESS_TOKEN: $(gh.access.token)
- stage: aur_update
dependsOn: build
condition: and(succeeded(), startsWith(variables['Build.SourceBranch'], 'refs/tags/'), not(endsWith(variables['Build.SourceBranch'], '-SNAPSHOT')), not(contains(variables['Build.SourceBranch'], '-b')))
jobs:
# Submit update to winget packages repository
- job: aur_update
pool:
vmImage: 'ubuntu-22.04'
steps:
- checkout: none
- task: DownloadSecureFile@1
name: aur_ssh_key_file
displayName: 'Download AUR ssh key'
inputs:
secureFile: 'aur_ssh_key_file'
# Install an SSH key prior to a build or deployment.
- task: InstallSSHKey@0
inputs:
knownHostsEntry: 'aur.archlinux.org ssh-ed25519 AAAAC3NzaC1lZDI1NTE5AAAAIEuBKrPzbawxA/k2g6NcyV5jmqwJ2s+zpgZGZ7tpLIcN'
sshPublicKey: '$(aur.ssh.public)'
sshPassphrase: '$(aur.ssh.password)'
sshKeySecureFile: 'aur_ssh_key_file'
# Advanced
addEntryToConfig: true
configHostAlias: 'aur.archlinux.org'
configHostname: 'aur.archlinux.org'
configUser: 'aur'
- bash: |
git clone ssh://aur@aur.archlinux.org/binjr-bin.git
cd binjr-bin
export OLD_VER=$(grep -Po 'pkgver=\K[^"]*' PKGBUILD)
export NEW_VER=${REPO_TAG_NAME//v}
# Change version and release number
sed -i "s/$OLD_VER/$NEW_VER/g" PKGBUILD
sed -i "s/^pkgrel=.*$/pkgrel=1/" PKGBUILD
sed -i "s/$OLD_VER/$NEW_VER/g" .SRCINFO
sed -i "s/pkgrel =.*/pkgrel = 1/" .SRCINFO
# Commit and push update
git config user.email "binjr.bot@free.fr"
git config user.name "binjr-bot"
git commit -am "Release ${REPO_TAG_NAME}-1"
git show
git push
env:
REPO_TAG_NAME: $(Build.SourceBranchName)
- stage: winget_update
dependsOn: build
condition: and(succeeded(), startsWith(variables['Build.SourceBranch'], 'refs/tags/'), not(endsWith(variables['Build.SourceBranch'], '-SNAPSHOT')), not(contains(variables['Build.SourceBranch'], '-b')))
jobs:
# Submit update to winget packages repository
- job: winget_update
pool:
vmImage: 'windows-2022'
steps:
- checkout: none
- task: PowerShell@2
displayName: install wingetCreate
inputs:
targetType: inline
script: |
# Download and install C++ Runtime framework package.
iwr https://aka.ms/Microsoft.VCLibs.x64.14.00.Desktop.appx -OutFile vcLibsBundleFile.appx
Add-AppxPackage vcLibsBundleFile.appx
# Download, install, and execute update.
iwr https://aka.ms/wingetcreate/latest/msixbundle -OutFile wingetcreate.appx
Add-AppxPackage wingetcreate.appx
- bash: |
wingetcreate.exe update binjr.core-x64 --urls https://github.com/binjr/binjr/releases/download/${REPO_TAG_NAME}/binjr-${REPO_TAG_NAME//v}_windows-amd64.msi --version ${REPO_TAG_NAME//v} --token $GH_ACCESS_TOKEN --submit
env:
GH_ACCESS_TOKEN: $(gh.access.token)
REPO_TAG_NAME: $(Build.SourceBranchName)
- stage: maven_publish
dependsOn: build
jobs:
# Publish to Maven repo
- job: maven_publish
pool:
vmImage: 'ubuntu-22.04'
steps:
- script: |
echo '##vso[task.setvariable variable=isTag;]true'
condition: and(succeeded(), startsWith(variables['Build.SourceBranch'], 'refs/tags/'))
- script: |
curl --output $(Agent.TempDirectory)/jdk-latest-linux_x64.tar.gz -O -J -L https://api.adoptium.net/v3/binary/latest/$(javaVersion)/ga/linux/x64/jdk/hotspot/normal/eclipse
- task: JavaToolInstaller@0
inputs:
versionSpec: $(javaVersion)
jdkArchitectureOption: x64
jdkSourceOption: LocalDirectory
jdkFile: $(Agent.TempDirectory)/jdk-latest-linux_x64.tar.gz
jdkDestinationDirectory: $(Agent.ToolsDirectory)/binaries/openjdk
cleanDestinationDirectory: true
- task: DownloadSecureFile@1
name: gpgKeyring
displayName: 'Download GPG Keyring'
inputs:
secureFile: 'keyring.gpg'
- task: Gradle@2
env:
IS_TAG: $(isTag)
REPO_TAG_NAME: $(Build.SourceBranchName)
BINJR_BUILD_NUMBER: $(Build.BuildNumber)
GPG_KEY_NAME: $(gpg.package.keyname)
GPG_PASSPHRASE: $(gpg.package.passphrase)
GPG_KEYRING_PATH: $(gpgKeyring.secureFilePath)
OSSRH_TOKEN_PASSWORD: $(ossrh.token.password)
OSSRH_TOKEN_USERNAME: $(ossrh.token.username)
inputs:
gradleWrapperFile: 'gradlew'
javaHomeOption: 'JDKVersion'
jdkVersionOption: 1.$(javaVersion)
jdkArchitectureOption: 'x64'
publishJUnitResults: false
tasks: 'publishArtifacts'
binjr/binjr-adapter-csv/README.md
+5
View File
@@ -0,0 +1,5 @@
# binjr-adapter-csv
[![Maven Central](https://img.shields.io/maven-central/v/eu.binjr/binjr-adapter-csv.svg?label=Maven%20Central&style=flat-square)](https://search.maven.org/search?q=g:%22eu.binjr%22%20AND%20a:%22binjr-adapter-csv%22)
This module implements a DataAdapter capable of consuming data from Comma Separated Values (i.e. CSV) files.
binjr/binjr-adapter-csv/build.gradle
+31
View File
@@ -0,0 +1,31 @@
/*
* Copyright 2018_2021 Frederic Thevenet
*
* Licensed under the Apache License, Version 2.0 (the "License");
* you may not use this file except in compliance with the License.
* You may obtain a copy of the License at
*
* http://www.apache.org/licenses/LICENSE-2.0
*
* Unless required by applicable law or agreed to in writing, software
* distributed under the License is distributed on an "AS IS" BASIS,
* WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
* See the License for the specific language governing permissions and
* limitations under the License.
*/
dependencies {
compileOnly project(':binjr-core')
}
jar {
manifest {
attributes(
'Specification-Title': project.name,
'Specification-Version': project.version,
'Implementation-Title': project.name,
'Implementation-Version': project.version,
'Build-Number': BINJR_BUILD_NUMBER
)
}
}
binjr/binjr-adapter-csv/src/main/java/eu/binjr/sources/csv/adapters/CsvAdapterPreferences.java
+89
View File
@@ -0,0 +1,89 @@
/*
* Copyright 2022 Frederic Thevenet
*
* Licensed under the Apache License, Version 2.0 (the "License");
* you may not use this file except in compliance with the License.
* You may obtain a copy of the License at
*
* http://www.apache.org/licenses/LICENSE-2.0
*
* Unless required by applicable law or agreed to in writing, software
* distributed under the License is distributed on an "AS IS" BASIS,
* WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
* See the License for the specific language governing permissions and
* limitations under the License.
*/
package eu.binjr.sources.csv.adapters;
import com.google.gson.Gson;
import eu.binjr.common.auth.JfxKrb5LoginModule;
import eu.binjr.common.preferences.ObservablePreference;
import eu.binjr.core.data.adapters.DataAdapter;
import eu.binjr.core.data.adapters.DataAdapterPreferences;
import eu.binjr.core.data.indexes.parser.profile.CustomParsingProfile;
import eu.binjr.core.data.indexes.parser.profile.ParsingProfile;
import eu.binjr.sources.csv.data.parsers.BuiltInCsvParsingProfile;
import eu.binjr.sources.csv.data.parsers.CsvParsingProfile;
import eu.binjr.sources.csv.data.parsers.CustomCsvParsingProfile;
/**
* Defines the preferences associated with the Log files adapter.
*
* @author Frederic Thevenet
*/
public class CsvAdapterPreferences extends DataAdapterPreferences {
private static final Gson gson = new Gson();
/**
* The default text panel font size preference.
*/
public ObservablePreference<Number> defaultTextViewFontSize = integerPreference("defaultTextViewFontSize", 10);
/**
* The filters used when scanning folders in the source filesystem.
*/
public ObservablePreference<String[]> folderFilters = objectPreference(String[].class,
"folderFilters",
new String[]{"*"},
gson::toJson,
s -> gson.fromJson(s, String[].class));
/**
* The filters used to prune file extensions to scan in the source filesystem.
*/
public ObservablePreference<String[]> fileExtensionFilters = objectPreference(String[].class,
"extensionFilters",
new String[]{"*.log", "*.txt"},
gson::toJson,
s -> gson.fromJson(s, String[].class));
/**
* The most recently used {@link ParsingProfile}
*/
public ObservablePreference<String> mostRecentlyUsedParsingProfile =
stringPreference("mruCsvParsingProfile", BuiltInCsvParsingProfile.ISO.getProfileId());
public ObservablePreference<CsvParsingProfile[]> csvTimestampParsingProfiles =
objectPreference(CsvParsingProfile[].class,
"csvTimestampParsingProfiles",
new CsvParsingProfile[0],
s -> gson.toJson(s),
s -> gson.fromJson(s, CustomCsvParsingProfile[].class)
);
public ObservablePreference<String> mruEncoding = stringPreference("mruEncoding", "utf-8");
public ObservablePreference<String> mruCsvSeparator = stringPreference("mruCsvSeparator", ";");
public ObservablePreference<Number> mruDateColumnPosition = integerPreference("mruDateColumnPosition", 0);
/**
* Initialize a new instance of the {@link CsvAdapterPreferences} class associated to
* a {@link DataAdapter} instance.
*
* @param dataAdapterClass the associated {@link DataAdapter}
*/
public CsvAdapterPreferences(Class<? extends DataAdapter<?>> dataAdapterClass) {
super(dataAdapterClass);
}
}
binjr/binjr-adapter-csv/src/main/java/eu/binjr/sources/csv/adapters/CsvFileAdapter.java
+317
View File
@@ -0,0 +1,317 @@
/*
* Copyright 2017-2022 Frederic Thevenet
*
* Licensed under the Apache License, Version 2.0 (the "License");
* you may not use this file except in compliance with the License.
* You may obtain a copy of the License at
*
* http://www.apache.org/licenses/LICENSE-2.0
*
* Unless required by applicable law or agreed to in writing, software
* distributed under the License is distributed on an "AS IS" BASIS,
* WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
* See the License for the specific language governing permissions and
* limitations under the License.
*/
package eu.binjr.sources.csv.adapters;
import com.google.gson.Gson;
import eu.binjr.common.function.CheckedLambdas;
import eu.binjr.common.io.FileSystemBrowser;
import eu.binjr.common.io.IOUtils;
import eu.binjr.common.javafx.controls.TimeRange;
import eu.binjr.common.logging.Logger;
import eu.binjr.core.data.adapters.*;
import eu.binjr.core.data.exceptions.CannotInitializeDataAdapterException;
import eu.binjr.core.data.exceptions.DataAdapterException;
import eu.binjr.core.data.exceptions.FetchingDataFromAdapterException;
import eu.binjr.core.data.exceptions.InvalidAdapterParameterException;
import eu.binjr.core.data.indexes.Index;
import eu.binjr.core.data.indexes.Indexes;
import eu.binjr.core.data.indexes.IndexingStatus;
import eu.binjr.core.data.timeseries.DoubleTimeSeriesProcessor;
import eu.binjr.core.data.timeseries.TimeSeriesProcessor;
import eu.binjr.core.data.workspace.TimeSeriesInfo;
import eu.binjr.core.data.workspace.XYChartsWorksheet;
import eu.binjr.sources.csv.data.parsers.BuiltInCsvParsingProfile;
import eu.binjr.sources.csv.data.parsers.CsvEventFormat;
import eu.binjr.sources.csv.data.parsers.CsvParsingProfile;
import eu.binjr.sources.csv.data.parsers.CustomCsvParsingProfile;
import javafx.beans.property.LongProperty;
import javafx.beans.property.Property;
import javafx.beans.property.SimpleLongProperty;
import javafx.beans.property.SimpleObjectProperty;
import javafx.scene.control.TreeItem;
import org.apache.lucene.document.StoredField;
import org.eclipse.fx.ui.controls.tree.FilterableTreeItem;
import java.io.IOException;
import java.io.InputStream;
import java.nio.charset.Charset;
import java.nio.file.Files;
import java.nio.file.Path;
import java.text.NumberFormat;
import java.time.Instant;
import java.time.ZoneId;
import java.util.HashMap;
import java.util.List;
import java.util.Map;
import java.util.Set;
import java.util.stream.Collectors;
/**
* A {@link DataAdapter} implementation used to feed {@link XYChartsWorksheet} instances
* with data from a local CSV formatted file.
*
* @author Frederic Thevenet
*/
public class CsvFileAdapter extends BaseDataAdapter<Double> {
private static final Logger logger = Logger.create(CsvFileAdapter.class);
private static final Gson gson = new Gson();
private static final Property<IndexingStatus> INDEXING_OK = new SimpleObjectProperty<>(IndexingStatus.OK);
private static final String ZONE_ID = "zoneId";
private static final String ENCODING = "encoding";
private static final String DELIMITER = "delimiter";
private static final String PARSING_PROFILE = "parsingProfile";
private static final String PATH = "csvPath";
private static final String TIMESTAMP_POSITION = "timestampPosition";
private CsvEventFormat parser;
private CsvParsingProfile csvParsingProfile;
private Path csvPath;
private ZoneId zoneId;
private String encoding;
private final Map<String, IndexingStatus> indexedFiles = new HashMap<>();
private Index index;
private FileSystemBrowser fileBrowser;
private String[] folderFilters;
private String[] fileExtensionsFilters;
private List<String> headers;
private long sequence = 0;
/**
* Initializes a new instance of the {@link CsvFileAdapter} class with a set of default values.
*
* @throws DataAdapterException if the {@link DataAdapter} could not be initializes.
*/
public CsvFileAdapter() throws DataAdapterException {
this("", ZoneId.systemDefault(), "utf-8", BuiltInCsvParsingProfile.ISO);
}
/**
* Initializes a new instance of the {@link CsvFileAdapter} class for the provided file and time zone.
*
* @param csvPath the path to the csv file.
* @param zoneId the time zone to used.
* @throws DataAdapterException if the {@link DataAdapter} could not be initialized.
*/
public CsvFileAdapter(String csvPath, ZoneId zoneId) throws DataAdapterException {
this(csvPath, zoneId, "utf-8", BuiltInCsvParsingProfile.ISO);
}
/**
* Initializes a new instance of the {@link CsvFileAdapter} class with the provided parameters.
*
* @param csvPath the path to the csv file.
* @param zoneId the time zone to used.
* @param encoding the encoding for the csv file.
* @param csvParsingProfile a pattern to decode time stamps.
* @throws DataAdapterException if the {@link DataAdapter} could not be initialized.
*/
public CsvFileAdapter(String csvPath,
ZoneId zoneId,
String encoding,
CsvParsingProfile csvParsingProfile)
throws DataAdapterException {
super();
initParams(zoneId, csvPath, encoding, csvParsingProfile);
}
@Override
public FilterableTreeItem<SourceBinding> getBindingTree() throws DataAdapterException {
FilterableTreeItem<SourceBinding> tree = new FilterableTreeItem<>(
new TimeSeriesBinding.Builder()
.withLabel(getSourceName())
.withPath("/")
.withAdapter(this)
.build());
try (InputStream in = Files.newInputStream(csvPath)) {
this.headers = parser.getDataColumnHeaders(in);
for (int i = 0; i < headers.size(); i++) {
if (i != csvParsingProfile.getTimestampColumn()) {
String header = headers.get(i).isBlank() ? "Column #" + i : headers.get(i);
var b = new TimeSeriesBinding.Builder()
.withLabel(Integer.toString(i))
.withPath(getId() + "/" + csvPath.toString())
.withLegend(header)
.withParent(tree.getValue())
.withAdapter(this)
.build();
tree.getInternalChildren().add(new TreeItem<>(b));
}
}
} catch (IOException e) {
throw new FetchingDataFromAdapterException(e);
}
return tree;
}
@Override
public TimeRange getInitialTimeRange(String path, List<TimeSeriesInfo<Double>> seriesInfo) throws DataAdapterException {
try {
ensureIndexed(seriesInfo.stream().map(TimeSeriesInfo::getBinding).collect(Collectors.toSet()), ReloadPolicy.UNLOADED);
return index.getTimeRangeBoundaries(seriesInfo.stream().map(ts -> ts.getBinding().getPath()).toList(), getTimeZoneId());
} catch (IOException e) {
throw new DataAdapterException("Error retrieving initial time range", e);
}
}
@Override
public Map<TimeSeriesInfo<Double>, TimeSeriesProcessor<Double>> fetchData(String path,
Instant begin,
Instant end,
List<TimeSeriesInfo<Double>> seriesInfo,
boolean bypassCache) throws DataAdapterException {
try {
ensureIndexed(seriesInfo.stream().map(TimeSeriesInfo::getBinding).collect(Collectors.toSet()), ReloadPolicy.UNLOADED);
Map<TimeSeriesInfo<Double>, TimeSeriesProcessor<Double>> series = new HashMap<>();
for (TimeSeriesInfo<Double> info : seriesInfo) {
series.put(info, new DoubleTimeSeriesProcessor());
}
var nbHits = index.search(
begin.toEpochMilli(),
end.toEpochMilli(),
series,
zoneId,
bypassCache);
logger.debug(() -> "Retrieved " + nbHits + " hits");
return series;
} catch (Exception e) {
throw new DataAdapterException("Error fetching data from " + path, e);
}
}
@Override
public String getEncoding() {
return encoding;
}
@Override
public ZoneId getTimeZoneId() {
return zoneId;
}
@Override
public String getSourceName() {
return new StringBuilder("[CSV] ")
.append(csvPath != null ? csvPath.getFileName() : "???")
.append(" (")
.append(zoneId != null ? zoneId : "???")
.append(")")
.toString();
}
@Override
public Map<String, String> getParams() {
Map<String, String> params = new HashMap<>();
params.put(ZONE_ID, zoneId.toString());
params.put(ENCODING, encoding);
params.put(PARSING_PROFILE, gson.toJson(CustomCsvParsingProfile.of(csvParsingProfile)));
params.put(PATH, csvPath.toString());
return params;
}
@Override
public void loadParams(Map<String, String> params) throws DataAdapterException {
if (params == null) {
throw new InvalidAdapterParameterException("Could not find parameter list for adapter " + getSourceName());
}
initParams(validateParameter(params, ZONE_ID,
s -> {
if (s == null) {
throw new InvalidAdapterParameterException("Parameter '" + ZONE_ID + "' is missing in adapter " + getSourceName());
}
return ZoneId.of(s);
}),
validateParameterNullity(params, PATH),
validateParameterNullity(params, ENCODING),
gson.fromJson(validateParameterNullity(params, PARSING_PROFILE), CustomCsvParsingProfile.class));
}
private void initParams(ZoneId zoneId,
String csvPath,
String encoding,
CsvParsingProfile parsingProfile) {
this.zoneId = zoneId;
this.csvPath = Path.of(csvPath);
this.encoding = encoding;
this.csvParsingProfile = parsingProfile;
this.parser = new CsvEventFormat(parsingProfile, zoneId, Charset.forName(encoding));
}
@Override
public void onStart() throws DataAdapterException {
super.onStart();
try {
this.fileBrowser = FileSystemBrowser.of(csvPath.getParent());
this.index = Indexes.NUM_SERIES.acquire();
} catch (IOException e) {
throw new CannotInitializeDataAdapterException("An error occurred during the data adapter initialization", e);
}
}
@Override
public void close() {
try {
Indexes.NUM_SERIES.release();
} catch (Exception e) {
logger.error("An error occurred while releasing index " + Indexes.NUM_SERIES.name() + ": " + e.getMessage());
logger.debug("Stack Trace:", e);
}
IOUtils.close(fileBrowser);
super.close();
}
private double formatToDouble(String value, NumberFormat numberFormat) {
if (value != null) {
try {
return numberFormat.parse(value).doubleValue();
} catch (Exception e) {
logger.trace(() -> "Failed to convert '" + value + "' to double");
}
}
return Double.NaN;
}
private synchronized void ensureIndexed(Set<SourceBinding<Double>> bindings, ReloadPolicy reloadPolicy) throws IOException {
if (reloadPolicy == ReloadPolicy.ALL) {
bindings.stream().map(SourceBinding::getPath).forEach(indexedFiles::remove);
}
final LongProperty charRead = new SimpleLongProperty(0);
for (var binding : bindings) {
String path = binding.getPath();
indexedFiles.computeIfAbsent(path, CheckedLambdas.wrap(p -> {
ThreadLocal<NumberFormat> formatters =
ThreadLocal.withInitial(() -> NumberFormat.getNumberInstance(csvParsingProfile.getNumberFormattingLocale()));
try {
index.add(p,
fileBrowser.getData(path.replace(getId() + "/", "")),
true,
parser,
(doc, event) -> {
event.getTextFields().forEach((key, value) -> doc.add(new StoredField(key, formatToDouble(value, formatters.get()))));
return doc;
},
charRead,
INDEXING_OK);
return IndexingStatus.OK;
} finally {
formatters.remove();
}
}));
}
}
}
binjr/binjr-adapter-csv/src/main/java/eu/binjr/sources/csv/adapters/CsvFileAdapterDialog.java
+171
View File
@@ -0,0 +1,171 @@
/*
* Copyright 2017-2023 Frederic Thevenet
*
* Licensed under the Apache License, Version 2.0 (the "License");
* you may not use this file except in compliance with the License.
* You may obtain a copy of the License at
*
* http://www.apache.org/licenses/LICENSE-2.0
*
* Unless required by applicable law or agreed to in writing, software
* distributed under the License is distributed on an "AS IS" BASIS,
* WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
* See the License for the specific language governing permissions and
* limitations under the License.
*/
package eu.binjr.sources.csv.adapters;
import eu.binjr.common.javafx.controls.LabelWithInlineHelp;
import eu.binjr.common.javafx.controls.NodeUtils;
import eu.binjr.core.data.adapters.DataAdapter;
import eu.binjr.core.data.adapters.DataAdapterFactory;
import eu.binjr.core.data.exceptions.CannotInitializeDataAdapterException;
import eu.binjr.core.data.exceptions.DataAdapterException;
import eu.binjr.core.data.exceptions.NoAdapterFoundException;
import eu.binjr.core.dialogs.DataAdapterDialog;
import eu.binjr.core.dialogs.Dialogs;
import eu.binjr.sources.csv.data.parsers.BuiltInCsvParsingProfile;
import eu.binjr.sources.csv.data.parsers.CsvParsingProfile;
import javafx.geometry.HPos;
import javafx.geometry.Insets;
import javafx.geometry.Pos;
import javafx.geometry.VPos;
import javafx.scene.Node;
import javafx.scene.control.*;
import javafx.scene.layout.GridPane;
import javafx.scene.layout.HBox;
import javafx.scene.layout.Priority;
import javafx.stage.FileChooser;
import org.controlsfx.control.textfield.TextFields;
import java.io.File;
import java.nio.charset.Charset;
import java.nio.file.Files;
import java.nio.file.Path;
import java.nio.file.Paths;
import java.time.ZoneId;
import java.util.Collection;
import java.util.List;
import java.util.Objects;
/**
* An implementation of the {@link DataAdapterDialog} class that presents a dialog box to retrieve the parameters specific {@link CsvFileAdapter}
*
* @author Frederic Thevenet
*/
public class CsvFileAdapterDialog extends DataAdapterDialog<Path> {
private final TextField encodingField;
private int pos = 2;
private final CsvAdapterPreferences prefs;
private final ChoiceBox<CsvParsingProfile> parsingChoiceBox = new ChoiceBox<>();
/**
* Initializes a new instance of the {@link CsvFileAdapterDialog} class.
*
* @param owner the owner window for the dialog
*/
public CsvFileAdapterDialog(Node owner) throws NoAdapterFoundException {
super(owner, Mode.PATH, "mostRecentCsvFiles", true);
this.prefs = (CsvAdapterPreferences) DataAdapterFactory.getInstance().getAdapterPreferences(CsvFileAdapter.class.getName());
this.setUriLabelInlineHelp("""
The path of the CSV file.
""");
this.setTimezoneLabelInlineHelp("""
The default timezone for timestamps in the CSV file.
""");
this.setDialogHeaderText("Add a csv file");
this.encodingField = new TextField(prefs.mruEncoding.get());
TextFields.bindAutoCompletion(encodingField, Charset.availableCharsets().keySet());
addParamField(this.encodingField, "Encoding", """
The text encoding of the CSV file.
""");
addParsingField(owner);
}
private void addParsingField(Node owner) {
var parsingLabel = new LabelWithInlineHelp("Parsing profile", """
The parsing profile used to process timestamps in the CSV file.
""");
parsingLabel.setAlignment(Pos.CENTER_RIGHT);
var parsingHBox = new HBox();
parsingHBox.setSpacing(5);
updateProfileList(prefs.csvTimestampParsingProfiles.get());
parsingChoiceBox.setMaxWidth(Double.MAX_VALUE);
var editParsingButton = new Button("Edit");
editParsingButton.setOnAction(event -> {
try {
new CsvParsingProfileDialog(this.getOwner(), parsingChoiceBox.getValue()).showAndWait().ifPresent(selection -> {
prefs.mostRecentlyUsedParsingProfile.set(selection.getProfileId());
updateProfileList(prefs.csvTimestampParsingProfiles.get());
});
} catch (Exception e) {
Dialogs.notifyException("Failed to show parsing profile windows", e, owner);
}
});
parsingHBox.getChildren().addAll(parsingChoiceBox, editParsingButton);
HBox.setHgrow(parsingChoiceBox, Priority.ALWAYS);
GridPane.setConstraints(parsingLabel, 0, pos, 1, 1, HPos.LEFT, VPos.CENTER, Priority.ALWAYS, Priority.ALWAYS, new Insets(4, 0, 4, 0));
GridPane.setConstraints(parsingHBox, 1, pos, 1, 1, HPos.LEFT, VPos.CENTER, Priority.ALWAYS, Priority.ALWAYS, new Insets(4, 0, 4, 0));
getParamsGridPane().getChildren().addAll(parsingLabel, parsingHBox);
pos++;
}
private void addParamField(TextField field, String label, String inlineHelp) {
GridPane.setConstraints(field, 1, pos, 1, 1, HPos.LEFT, VPos.CENTER, Priority.ALWAYS, Priority.ALWAYS, new Insets(4, 0, 4, 0));
var tabsLabel = new LabelWithInlineHelp(label, inlineHelp);
tabsLabel.setAlignment(Pos.CENTER_RIGHT);
GridPane.setConstraints(tabsLabel, 0, pos, 1, 1, HPos.LEFT, VPos.CENTER, Priority.ALWAYS, Priority.ALWAYS, new Insets(4, 0, 4, 0));
getParamsGridPane().getChildren().addAll(field, tabsLabel);
pos++;
}
@Override
protected File displayFileChooser(Node owner) {
try {
FileChooser fileChooser = new FileChooser();
fileChooser.setTitle("Open CSV file");
fileChooser.getExtensionFilters().add(new FileChooser.ExtensionFilter("Comma-separated values files", "*.csv"));
fileChooser.getExtensionFilters().add(new FileChooser.ExtensionFilter("All files", "*.*", "*"));
Dialogs.getInitialDir(getMostRecentList()).ifPresent(fileChooser::setInitialDirectory);
return fileChooser.showOpenDialog(NodeUtils.getStage(owner));
} catch (Exception e) {
Dialogs.notifyException("Error while displaying file chooser: " + e.getMessage(), e, owner);
}
return null;
}
@Override
protected Collection<DataAdapter> getDataAdapters() throws DataAdapterException {
Path csvPath = Paths.get(getSourceUri());
if (!Files.exists(csvPath)) {
throw new CannotInitializeDataAdapterException("Cannot find " + getSourceUri());
}
if (!csvPath.isAbsolute()) {
throw new CannotInitializeDataAdapterException("The provided path is not valid.");
}
getMostRecentList().push(csvPath);
prefs.mostRecentlyUsedParsingProfile.set(parsingChoiceBox.getValue().getProfileId());
String charsetName = encodingField.getText();
if (!Charset.isSupported(charsetName)) {
throw new CannotInitializeDataAdapterException("Invalid or unsupported encoding: " + charsetName);
}
prefs.mruEncoding.set(charsetName);
return List.of(new CsvFileAdapter(
getSourceUri(),
ZoneId.of(getSourceTimezone()),
charsetName,
parsingChoiceBox.getValue()));
}
private void updateProfileList(CsvParsingProfile[] newValue) {
parsingChoiceBox.getItems().clear();
parsingChoiceBox.getItems().setAll(BuiltInCsvParsingProfile.values());
parsingChoiceBox.getItems().addAll(newValue);
parsingChoiceBox.getSelectionModel().select(parsingChoiceBox.getItems().stream()
.filter(p -> Objects.equals(p.getProfileId(), prefs.mostRecentlyUsedParsingProfile.get()))
.findAny().orElse(BuiltInCsvParsingProfile.ISO));
}
}
binjr/binjr-adapter-csv/src/main/java/eu/binjr/sources/csv/adapters/CsvFileDataAdapterInfo.java
+53
View File
@@ -0,0 +1,53 @@
/*
* Copyright 2017-2023 Frederic Thevenet
*
* Licensed under the Apache License, Version 2.0 (the "License");
* you may not use this file except in compliance with the License.
* You may obtain a copy of the License at
*
* http://www.apache.org/licenses/LICENSE-2.0
*
* Unless required by applicable law or agreed to in writing, software
* distributed under the License is distributed on an "AS IS" BASIS,
* WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
* See the License for the specific language governing permissions and
* limitations under the License.
*/
package eu.binjr.sources.csv.adapters;
import eu.binjr.core.data.adapters.AdapterMetadata;
import eu.binjr.core.data.adapters.BaseDataAdapterInfo;
import eu.binjr.core.data.adapters.SourceLocality;
import eu.binjr.core.data.adapters.VisualizationType;
import eu.binjr.core.data.exceptions.CannotInitializeDataAdapterException;
import eu.binjr.core.preferences.AppEnvironment;
/**
* Defines the metadata associated with the CsvFileDataAdapter.
*
* @author Frederic Thevenet
*/
@AdapterMetadata(
name = "CSV",
description = "CSV File Data Adapter",
copyright = AppEnvironment.COPYRIGHT_NOTICE,
license = AppEnvironment.LICENSE,
siteUrl = AppEnvironment.HTTP_WWW_BINJR_EU,
adapterClass = CsvFileAdapter.class,
dialogClass = CsvFileAdapterDialog.class,
preferencesClass = CsvAdapterPreferences.class,
sourceLocality = SourceLocality.LOCAL,
apiLevel = AppEnvironment.PLUGIN_API_LEVEL,
visualizationType = VisualizationType.CHARTS
)
public class CsvFileDataAdapterInfo extends BaseDataAdapterInfo {
/**
* Initialises a new instance of the {@link CsvFileDataAdapterInfo} class.
* @throws CannotInitializeDataAdapterException if the adapter's initialization failed
*/
public CsvFileDataAdapterInfo() throws CannotInitializeDataAdapterException {
super(CsvFileDataAdapterInfo.class);
}
}
binjr/binjr-adapter-csv/src/main/java/eu/binjr/sources/csv/adapters/CsvParsingProfileDialog.java
+107
View File
@@ -0,0 +1,107 @@
/*
* Copyright 2022 Frederic Thevenet
*
* Licensed under the Apache License, Version 2.0 (the "License");
* you may not use this file except in compliance with the License.
* You may obtain a copy of the License at
*
* http://www.apache.org/licenses/LICENSE-2.0
*
* Unless required by applicable law or agreed to in writing, software
* distributed under the License is distributed on an "AS IS" BASIS,
* WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
* See the License for the specific language governing permissions and
* limitations under the License.
*/
package eu.binjr.sources.csv.adapters;
import eu.binjr.common.javafx.bindings.BindingManager;
import eu.binjr.common.javafx.controls.NodeUtils;
import eu.binjr.core.appearance.StageAppearanceManager;
import eu.binjr.core.preferences.UserPreferences;
import eu.binjr.sources.csv.data.parsers.CsvParsingProfile;
import eu.binjr.sources.csv.data.parsers.CsvParsingProfilesController;
import eu.binjr.core.data.adapters.DataAdapterFactory;
import eu.binjr.core.data.exceptions.NoAdapterFoundException;
import eu.binjr.sources.csv.data.parsers.BuiltInCsvParsingProfile;
import javafx.event.ActionEvent;
import javafx.fxml.FXMLLoader;
import javafx.scene.control.*;
import javafx.scene.input.KeyCode;
import javafx.scene.input.KeyEvent;
import javafx.stage.Modality;
import javafx.stage.StageStyle;
import javafx.stage.Window;
import java.io.IOException;
import java.nio.charset.StandardCharsets;
import java.time.ZoneId;
public class CsvParsingProfileDialog extends Dialog<CsvParsingProfile> {
private final DialogPane root;
public CsvParsingProfileDialog(Window owner, CsvParsingProfile selectedProfile) throws NoAdapterFoundException {
FXMLLoader fXMLLoader = new FXMLLoader(getClass().getResource("/eu/binjr/views/CsvParsingProfileDialogView.fxml"));
final CsvAdapterPreferences prefs;
prefs = (CsvAdapterPreferences) DataAdapterFactory.getInstance().getAdapterPreferences(CsvFileAdapter.class.getName());
var controller = new CsvParsingProfilesController(
BuiltInCsvParsingProfile.values(),
prefs.csvTimestampParsingProfiles.get(),
BuiltInCsvParsingProfile.ISO,
selectedProfile,
true,
StandardCharsets.UTF_8,
ZoneId.systemDefault());
fXMLLoader.setController(controller);
try {
root = fXMLLoader.load();
} catch (IOException e) {
throw new IllegalArgumentException("Failed to load " + fXMLLoader.getLocation());
}
this.setTitle("Edit CSV Parsing Profile");
this.setDialogPane(root);
this.setResizable(true);
this.initOwner(owner);
this.initStyle(StageStyle.UTILITY);
this.initModality(Modality.APPLICATION_MODAL);
BindingManager manager = new BindingManager();
var stage = NodeUtils.getStage(root);
stage.setUserData(manager);
stage.addEventFilter(KeyEvent.KEY_PRESSED, manager.registerHandler(e -> {
if (e.getCode() == KeyCode.F1) {
UserPreferences.getInstance().showInlineHelpButtons.set(!UserPreferences.getInstance().showInlineHelpButtons.get());
e.consume();
}
}));
this.setOnCloseRequest(event -> manager.registerHandler(e -> manager.close()));
StageAppearanceManager.getInstance().register(NodeUtils.getStage(root),
StageAppearanceManager.AppearanceOptions.SET_ICON,
StageAppearanceManager.AppearanceOptions.SET_THEME);
Button okButton = (Button) getDialogPane().lookupButton(ButtonType.OK);
okButton.addEventFilter(ActionEvent.ACTION, ae -> {
if (!controller.applyChanges()) {
ae.consume();
}
});
this.setResultConverter(dialogButton -> {
ButtonBar.ButtonData data = dialogButton == null ? null : dialogButton.getButtonData();
if (data == ButtonBar.ButtonData.OK_DONE) {
prefs.csvTimestampParsingProfiles.set(controller.getCustomProfiles().toArray(CsvParsingProfile[]::new));
return controller.getSelectedProfile();
}
return null;
}
);
}
}
binjr/binjr-adapter-csv/src/main/java/eu/binjr/sources/csv/data/parsers/BuiltInCsvParsingProfile.java
+179
View File
@@ -0,0 +1,179 @@
/*
* Copyright 2022 Frederic Thevenet
*
* Licensed under the Apache License, Version 2.0 (the "License");
* you may not use this file except in compliance with the License.
* You may obtain a copy of the License at
*
* http://www.apache.org/licenses/LICENSE-2.0
*
* Unless required by applicable law or agreed to in writing, software
* distributed under the License is distributed on an "AS IS" BASIS,
* WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
* See the License for the specific language governing permissions and
* limitations under the License.
*/
package eu.binjr.sources.csv.data.parsers;
import eu.binjr.core.data.indexes.parser.capture.CaptureGroup;
import eu.binjr.core.data.indexes.parser.capture.NamedCaptureGroup;
import eu.binjr.core.data.indexes.parser.capture.TemporalCaptureGroup;
import eu.binjr.core.data.indexes.parser.profile.ParsingProfile;
import java.text.NumberFormat;
import java.util.Locale;
import java.util.Map;
import java.util.regex.Pattern;
public enum BuiltInCsvParsingProfile implements CsvParsingProfile {
ISO("ISO timestamps",
"BUILTIN_ISO",
Map.of(TemporalCaptureGroup.YEAR, "\\d{4}",
TemporalCaptureGroup.MONTH, "\\d{2}",
TemporalCaptureGroup.DAY, "\\d{2}",
TemporalCaptureGroup.HOUR, "\\d{2}",
TemporalCaptureGroup.MINUTE, "\\d{2}",
TemporalCaptureGroup.SECOND, "\\d{2}",
TemporalCaptureGroup.MILLI, "\\d{3}",
CaptureGroup.of("TIMEZONE"), "(Z|[+-]\\d{2}:?(\\d{2})?)"),
"$YEAR[\\s\\/-]$MONTH[\\s\\/-]$DAY([-\\sT]$HOUR:$MINUTE:$SECOND)?([\\.,]$MILLI)?$TIMEZONE?",
",",
'"',
0,
new int[0],
true,
Locale.US,
false),
EPOCH("Seconds since 01/01/1970",
"EPOCH",
Map.of(TemporalCaptureGroup.EPOCH, "\\d+"),
"$EPOCH",
",",
'"',
0,
new int[0],
true,
Locale.US,
false),
EPOCH_MS("Milliseconds since 01/01/1970",
"EPOCH_MS",
Map.of(TemporalCaptureGroup.EPOCH, "\\d+",
TemporalCaptureGroup.MILLI, "\\d{3}"),
"$EPOCH$MILLI",
",",
'"',
0,
new int[0],
true,
Locale.US,
false);
private final String profileName;
private final String lineTemplateExpression;
private final Map<NamedCaptureGroup, String> captureGroups;
private final String profileId;
private final Pattern regex;
private final String delimiter;
private final int timestampColumn;
private final int[] excludedColumns;
private final Locale numberFormattingLocale;
private final boolean readColumnNames;
private final char quoteCharacter;
private final boolean trimCellValues;
BuiltInCsvParsingProfile(String profileName,
String id,
Map<NamedCaptureGroup, String> groups,
String lineTemplateExpression,
String delimiter,
char quoteChar,
int timestampColumn,
int[] excludedColumns,
boolean readColumnNames,
Locale numberFormattingLocale,
boolean trimCellValues) {
this.profileId = id;
this.profileName = profileName;
this.captureGroups = groups;
this.lineTemplateExpression = lineTemplateExpression;
this.regex = Pattern.compile(buildParsingRegexString());
this.delimiter = delimiter;
this.quoteCharacter = quoteChar;
this.timestampColumn = timestampColumn;
this.excludedColumns = excludedColumns;
this.readColumnNames = readColumnNames;
this.numberFormattingLocale = numberFormattingLocale;
this.trimCellValues = trimCellValues;
}
@Override
public String getLineTemplateExpression() {
return lineTemplateExpression;
}
@Override
public Pattern getParsingRegex() {
return regex;
}
@Override
public boolean isBuiltIn() {
return true;
}
@Override
public String getProfileId() {
return this.profileId;
}
@Override
public String getProfileName() {
return profileName;
}
@Override
public Map<NamedCaptureGroup, String> getCaptureGroups() {
return captureGroups;
}
@Override
public String toString() {
return profileName;
}
@Override
public String getDelimiter() {
return delimiter;
}
@Override
public int getTimestampColumn() {
return timestampColumn;
}
@Override
public int[] getExcludedColumns() {
return excludedColumns;
}
@Override
public boolean isReadColumnNames() {
return readColumnNames;
}
@Override
public Locale getNumberFormattingLocale() {
return numberFormattingLocale;
}
@Override
public char getQuoteCharacter() {
return quoteCharacter;
}
@Override
public boolean isTrimCellValues() {
return trimCellValues;
}
}
binjr/binjr-adapter-csv/src/main/java/eu/binjr/sources/csv/data/parsers/CsvEventFormat.java
+92
View File
@@ -0,0 +1,92 @@
/*
* Copyright 2020-2022 Frederic Thevenet
*
* Licensed under the Apache License, Version 2.0 (the "License");
* you may not use this file except in compliance with the License.
* You may obtain a copy of the License at
*
* http://www.apache.org/licenses/LICENSE-2.0
*
* Unless required by applicable law or agreed to in writing, software
* distributed under the License is distributed on an "AS IS" BASIS,
* WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
* See the License for the specific language governing permissions and
* limitations under the License.
*/
package eu.binjr.sources.csv.data.parsers;
import eu.binjr.common.logging.Logger;
import eu.binjr.common.text.StringUtils;
import eu.binjr.core.data.exceptions.DecodingDataFromAdapterException;
import eu.binjr.core.data.indexes.parser.EventFormat;
import eu.binjr.core.data.indexes.parser.EventParser;
import org.apache.commons.csv.CSVFormat;
import org.apache.commons.csv.CSVRecord;
import java.io.*;
import java.nio.charset.Charset;
import java.time.ZoneId;
import java.util.*;
public class CsvEventFormat implements EventFormat<InputStream> {
private static final Logger logger = Logger.create(CsvEventFormat.class);
private final CsvParsingProfile profile;
private final ZoneId zoneId;
private final Charset encoding;
public CsvEventFormat(CsvParsingProfile profile, ZoneId zoneId, Charset encoding) {
this.profile = profile;
this.zoneId = zoneId;
this.encoding = encoding;
}
@Override
public CsvParsingProfile getProfile() {
return profile;
}
@Override
public EventParser parse(InputStream ias) {
return new CsvEventParser(this, ias);
}
@Override
public Charset getEncoding() {
return encoding;
}
@Override
public ZoneId getZoneId() {
return zoneId;
}
/**
* Returns the columns headers of the CSV file.
*
* @param in an input stream for the CSV file.
* @return the columns headers of the CSV file.
* @throws IOException in the event of an I/O error.
* @throws DecodingDataFromAdapterException if an error occurred while decoding the CSV file.
*/
public List<String> getDataColumnHeaders(InputStream in) throws IOException, DecodingDataFromAdapterException {
try (BufferedReader reader = new BufferedReader(new InputStreamReader(in, encoding))) {
CSVFormat csvFormat = CSVFormat.Builder.create()
.setAllowMissingColumnNames(false)
.setDelimiter(StringUtils.stringToEscapeSequence(getProfile().getDelimiter()))
.build();
Iterable<CSVRecord> records = csvFormat.parse(reader);
CSVRecord record = records.iterator().next();
if (record == null) {
throw new DecodingDataFromAdapterException("CSV stream does not contains column header");
}
List<String> headerNames = new ArrayList<>();
for (int i = 0; i < record.size(); i++) {
String name = getProfile().isReadColumnNames() ? record.get(i) : "Column " + (i + 1);
headerNames.add(name);
}
return headerNames;
}
}
}
binjr/binjr-adapter-csv/src/main/java/eu/binjr/sources/csv/data/parsers/CsvEventParser.java
+147
View File
@@ -0,0 +1,147 @@
/*
* Copyright 2022-2023 Frederic Thevenet
*
* Licensed under the Apache License, Version 2.0 (the "License");
* you may not use this file except in compliance with the License.
* You may obtain a copy of the License at
*
* http://www.apache.org/licenses/LICENSE-2.0
*
* Unless required by applicable law or agreed to in writing, software
* distributed under the License is distributed on an "AS IS" BASIS,
* WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
* See the License for the specific language governing permissions and
* limitations under the License.
*/
package eu.binjr.sources.csv.data.parsers;
import eu.binjr.common.logging.Logger;
import eu.binjr.common.text.StringUtils;
import eu.binjr.core.data.indexes.parser.EventParser;
import eu.binjr.core.data.indexes.parser.ParsedEvent;
import eu.binjr.core.data.indexes.parser.capture.NamedCaptureGroup;
import eu.binjr.core.data.indexes.parser.capture.TemporalCaptureGroup;
import javafx.beans.property.LongProperty;
import javafx.beans.property.SimpleLongProperty;
import org.apache.commons.csv.CSVFormat;
import org.apache.commons.csv.CSVParser;
import java.io.BufferedReader;
import java.io.IOException;
import java.io.InputStream;
import java.io.InputStreamReader;
import java.time.ZonedDateTime;
import java.time.temporal.ChronoUnit;
import java.util.Iterator;
import java.util.LinkedHashMap;
import java.util.Map;
import java.util.concurrent.atomic.AtomicLong;
public class CsvEventParser implements EventParser {
private static final Logger logger = Logger.create(CsvEventParser.class);
private final BufferedReader reader;
private final AtomicLong sequence;
private final CsvEventFormat format;
private final CsvEventIterator eventIterator;
private final CSVParser csvParser;
private final LongProperty progress = new SimpleLongProperty(0);
CsvEventParser(CsvEventFormat format, InputStream ias) {
this.reader = new BufferedReader(new InputStreamReader(ias, format.getEncoding()));
this.sequence = new AtomicLong(0);
this.format = format;
try {
var builder = CSVFormat.Builder.create()
.setAllowMissingColumnNames(true)
.setSkipHeaderRecord(true)
.setTrim(format.getProfile().isTrimCellValues())
.setQuote(format.getProfile().getQuoteCharacter())
.setDelimiter(StringUtils.stringToEscapeSequence(format.getProfile().getDelimiter()));
if (format.getProfile().isReadColumnNames()) {
builder.setHeader();
}
this.csvParser = builder.build().parse(reader);
this.eventIterator = new CsvEventIterator();
} catch (IOException e) {
throw new RuntimeException(e);
}
}
@Override
public void close() throws IOException {
reader.close();
}
@Override
public LongProperty progressIndicator() {
return progress;
}
@Override
public Iterator<ParsedEvent> iterator() {
return eventIterator;
}
public class CsvEventIterator implements Iterator<ParsedEvent> {
@Override
public ParsedEvent next() {
var csvRecord = csvParser.iterator().next();
if (csvRecord == null) {
return null;
}
ZonedDateTime timestamp;
if (format.getProfile().getTimestampColumn() == -1) {
timestamp = ZonedDateTime.of(format.getProfile().getTemporalAnchor().resolve().plus(sequence.get(), ChronoUnit.SECONDS), format.getZoneId());
} else {
if (format.getProfile().getTimestampColumn() > csvRecord.size() - 1) {
throw new UnsupportedOperationException("Cannot extract time stamp in column #" +
(format.getProfile().getTimestampColumn() + 1) +
": CSV record only has " + csvRecord.size() + " fields.");
}
String dateString = csvRecord.get(format.getProfile().getTimestampColumn());
timestamp = parseDateTime(dateString);
}
if (timestamp == null) {
throw new UnsupportedOperationException("Failed to parse time stamp in column #" +
(format.getProfile().getTimestampColumn() + 1));
}
Map<String, String> values = new LinkedHashMap<>(csvRecord.size());
for (int i = 0; i < csvRecord.size(); i++) {
if (i != format.getProfile().getTimestampColumn()) {
// don't add the timestamp column as an attribute
values.put(Integer.toString(i), csvRecord.get(i));
}
}
return ParsedEvent.withTextFields(sequence.incrementAndGet(), timestamp, " ", values);
}
@Override
public boolean hasNext() {
return csvParser.iterator().hasNext();
}
private ZonedDateTime parseDateTime(String text) {
var m = format.getProfile().getParsingRegex().matcher(text);
ZonedDateTime timestamp = ZonedDateTime.of(format.getProfile().getTemporalAnchor().resolve(), format.getZoneId());
if (m.find()) {
for (Map.Entry<NamedCaptureGroup, String> entry : format.getProfile().getCaptureGroups().entrySet()) {
var captureGroup = entry.getKey();
var parsed = m.group(captureGroup.name());
if (parsed != null && !parsed.isBlank()) {
if (captureGroup instanceof TemporalCaptureGroup temporalGroup) {
timestamp = timestamp.with(temporalGroup.getMapping(), temporalGroup.parseLong(parsed));
}
}
}
return timestamp;
}
return null;
}
}
}
binjr/binjr-adapter-csv/src/main/java/eu/binjr/sources/csv/data/parsers/CsvParsingProfile.java
+40
View File
@@ -0,0 +1,40 @@
/*
* Copyright 2022 Frederic Thevenet
*
* Licensed under the Apache License, Version 2.0 (the "License");
* you may not use this file except in compliance with the License.
* You may obtain a copy of the License at
*
* http://www.apache.org/licenses/LICENSE-2.0
*
* Unless required by applicable law or agreed to in writing, software
* distributed under the License is distributed on an "AS IS" BASIS,
* WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
* See the License for the specific language governing permissions and
* limitations under the License.
*/
package eu.binjr.sources.csv.data.parsers;
import com.google.gson.annotations.JsonAdapter;
import eu.binjr.common.json.adapters.PatternJsonAdapter;
import eu.binjr.core.data.indexes.parser.profile.ParsingProfile;
import java.text.NumberFormat;
import java.util.Locale;
public interface CsvParsingProfile extends ParsingProfile {
String getDelimiter();
int getTimestampColumn();
int[] getExcludedColumns();
boolean isReadColumnNames();
Locale getNumberFormattingLocale();
char getQuoteCharacter();
boolean isTrimCellValues();
}
binjr/binjr-adapter-csv/src/main/java/eu/binjr/sources/csv/data/parsers/CsvParsingProfilesController.java
+341
View File
@@ -0,0 +1,341 @@
/*
* Copyright 2022 Frederic Thevenet
*
* Licensed under the Apache License, Version 2.0 (the "License");
* you may not use this file except in compliance with the License.
* You may obtain a copy of the License at
*
* http://www.apache.org/licenses/LICENSE-2.0
*
* Unless required by applicable law or agreed to in writing, software
* distributed under the License is distributed on an "AS IS" BASIS,
* WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
* See the License for the specific language governing permissions and
* limitations under the License.
*/
package eu.binjr.sources.csv.data.parsers;
import com.google.gson.reflect.TypeToken;
import eu.binjr.common.javafx.charts.StableTicksAxis;
import eu.binjr.common.javafx.controls.AlignedTableCellFactory;
import eu.binjr.common.javafx.controls.TextFieldValidator;
import eu.binjr.common.javafx.controls.ToolButtonBuilder;
import eu.binjr.common.text.StringUtils;
import eu.binjr.core.controllers.ParsingProfilesController;
import eu.binjr.core.data.adapters.DataAdapterFactory;
import eu.binjr.core.data.exceptions.NoAdapterFoundException;
import eu.binjr.core.data.indexes.parser.EventParser;
import eu.binjr.core.data.indexes.parser.ParsedEvent;
import eu.binjr.core.data.indexes.parser.capture.NamedCaptureGroup;
import eu.binjr.sources.csv.adapters.CsvAdapterPreferences;
import eu.binjr.sources.csv.adapters.CsvFileAdapter;
import javafx.beans.property.SimpleStringProperty;
import javafx.event.ActionEvent;
import javafx.fxml.FXML;
import javafx.scene.control.*;
import javafx.scene.text.TextAlignment;
import javafx.stage.FileChooser;
import org.controlsfx.control.textfield.TextFields;
import java.io.ByteArrayInputStream;
import java.io.InputStream;
import java.lang.reflect.Type;
import java.net.URL;
import java.nio.charset.Charset;
import java.text.NumberFormat;
import java.time.ZoneId;
import java.time.format.DateTimeFormatter;
import java.util.*;
import java.util.function.UnaryOperator;
public class CsvParsingProfilesController extends ParsingProfilesController<CsvParsingProfile> {
@FXML
private Spinner<ColumnPosition> timeColumnTextField;
@FXML
private TextField delimiterTextField;
@FXML
private TextField quoteCharacterTextField;
@FXML
private TableView<ParsedEvent> testResultTable;
@FXML
private TabPane testTabPane;
@FXML
private Tab inputTab;
@FXML
private Tab resultTab;
@FXML
private CheckBox readColumnNameCheckBox;
@FXML
private TextField parsingLocaleTextField;
@FXML
private CheckBox trimCellsCheckbox;
private final UnaryOperator<TextFormatter.Change> clampToSingleChar = c -> {
if (c.isContentChange()) {
String newText = c.getControlNewText();
int newLength = newText.length();
if (newLength > 1 && (newText.charAt(0) != '\\' || !List.of('t', 'r', 'n').contains(newText.charAt(1)) || newLength > 2)) {
String tail = newText.substring(newLength - 1, newLength);
c.setText(tail);
int oldLength = c.getControlText().length();
c.setRange(0, oldLength);
}
}
return c;
};
private NumberFormat numberFormat = NumberFormat.getNumberInstance();
@Override
public void initialize(URL location, ResourceBundle resources) {
super.initialize(location, resources);
delimiterTextField.textProperty().addListener((observable) -> resetTest());
timeColumnTextField.valueProperty().addListener(observable -> resetTest());
readColumnNameCheckBox.selectedProperty().addListener(observable -> resetTest());
trimCellsCheckbox.selectedProperty().addListener(observable -> resetTest());
TextFields.bindAutoCompletion(parsingLocaleTextField,
Arrays.stream(Locale.getAvailableLocales()).map(Locale::toLanguageTag).toList());
delimiterTextField.setTextFormatter(new TextFormatter<>(clampToSingleChar));
quoteCharacterTextField.setTextFormatter(new TextFormatter<>(clampToSingleChar));
}
@Override
protected void loadParserParameters(CsvParsingProfile profile) {
super.loadParserParameters(profile);
this.delimiterTextField.setText(profile.getDelimiter());
this.quoteCharacterTextField.setText(String.valueOf(profile.getQuoteCharacter()));
this.timeColumnTextField.setValueFactory(new ColumnPositionFactory(-1, 999999, profile.getTimestampColumn()));
this.readColumnNameCheckBox.setSelected(profile.isReadColumnNames());
this.parsingLocaleTextField.setText(profile.getNumberFormattingLocale().toLanguageTag());
this.trimCellsCheckbox.setSelected(profile.isTrimCellValues());
}
public record ColumnPosition(int index) {
@Override
public String toString() {
return (index < 0) ? "line numbers" : StringUtils.integerToOrdinal(index + 1) + " column";
}
}
public static class ColumnPositionFactory extends SpinnerValueFactory<ColumnPosition> {
private final int minValue;
private final int maxValue;
private int index = 0;
public ColumnPositionFactory(int minValue, int maxValue, int initialValue) {
if (initialValue > maxValue) {
throw new IllegalArgumentException("Initial value is above maximum value");
}
if (initialValue < minValue) {
throw new IllegalArgumentException("Initial value is below minimum value");
}
this.minValue = minValue;
this.maxValue = maxValue;
this.index = initialValue;
setValue(new ColumnPosition(index));
}
@Override
public void decrement(int steps) {
int newPos = index - steps;
if (newPos >= minValue) {
this.index = newPos;
setValue(new ColumnPosition(index));
}
}
@Override
public void increment(int steps) {
int newPos = index + steps;
if (newPos <= maxValue) {
this.index = newPos;
setValue(new ColumnPosition(index));
}
}
}
public CsvParsingProfilesController(CsvParsingProfile[] builtinParsingProfiles,
CsvParsingProfile[] userParsingProfiles,
CsvParsingProfile defaultProfile,
CsvParsingProfile selectedProfile,
Charset defaultCharset,
ZoneId defaultZoneId) throws NoAdapterFoundException {
this(builtinParsingProfiles,
userParsingProfiles,
defaultProfile,
selectedProfile,
true,
defaultCharset,
defaultZoneId);
}
public CsvParsingProfilesController(CsvParsingProfile[] builtinParsingProfiles,
CsvParsingProfile[] userParsingProfiles,
CsvParsingProfile defaultProfile,
CsvParsingProfile selectedProfile,
boolean allowTemporalCaptureGroupsOnly,
Charset defaultCharset,
ZoneId defaultZoneId) throws NoAdapterFoundException {
super(builtinParsingProfiles,
userParsingProfiles,
defaultProfile,
selectedProfile,
allowTemporalCaptureGroupsOnly,
defaultCharset,
defaultZoneId);
CsvAdapterPreferences prefs = (CsvAdapterPreferences) DataAdapterFactory.getInstance().getAdapterPreferences(CsvFileAdapter.class.getName());
}
@Override
protected void handleOnRunTest(ActionEvent event) {
super.handleOnRunTest(event);
testTabPane.getSelectionModel().select(resultTab);
}
@Override
protected Optional<List<FileChooser.ExtensionFilter>> additionalExtensions() {
return Optional.of(List.of(new FileChooser.ExtensionFilter("Comma-separated values files", "*.csv")));
}
@Override
protected List<CsvParsingProfile> deSerializeProfiles(String profileString) {
Type profileListType = new TypeToken<ArrayList<CustomCsvParsingProfile>>() {
}.getType();
return gson.fromJson(profileString, profileListType);
}
@Override
protected void doTest() throws Exception {
var format = new CsvEventFormat(profileComboBox.getValue(), getDefaultZoneId(), getDefaultCharset());
try (InputStream in = new ByteArrayInputStream(testArea.getText().getBytes(getDefaultCharset()))) {
var headers = format.getDataColumnHeaders(in);
if (headers.size() == 0) {
notifyWarn("No record found.");
} else {
Map<TableColumn, String> colMap = new HashMap<>();
var tsColNum = format.getProfile().getTimestampColumn();
// Add a column for line numbers
var lineNbColumn = makeColumn(colMap, TextAlignment.CENTER, -1, tsColNum, "#");
lineNbColumn.setCellValueFactory(param ->
new SimpleStringProperty(Long.toString(param.getValue().getSequence())));
lineNbColumn.getStyleClass().add("line-number-column");
testResultTable.getColumns().add(lineNbColumn);
for (int i = 0; i < headers.size(); i++) {
String name = headers.get(i);
TableColumn<ParsedEvent, String> col = makeColumn(colMap, TextAlignment.RIGHT, i, tsColNum, name);
if (i == tsColNum) {
col.setCellValueFactory(param ->
new SimpleStringProperty(param.getValue().getTimestamp().format(DateTimeFormatter.ofPattern("yyyy-MM-dd HH:mm:ss[.SSS]"))));
} else {
col.setCellValueFactory(param ->
new SimpleStringProperty(formatToDouble(param.getValue().getTextFields().get(colMap.get(param.getTableColumn())))));
}
testResultTable.getColumns().add(col);
}
}
}
try (InputStream in = new ByteArrayInputStream(testArea.getText().getBytes(getDefaultCharset()))) {
EventParser eventParser = format.parse(in);
for (ParsedEvent parsed : eventParser) {
testResultTable.getItems().add(parsed);
}
notifyInfo(String.format("Found %d record(s).", testResultTable.getItems().size()));
}
}
private TableColumn<ParsedEvent, String> makeColumn(Map<TableColumn, String> colMap,
TextAlignment alignment,
int i,
int tsColumn,
String name) {
var col = new TableColumn<ParsedEvent, String>(name);
col.setStyle("-fx-font-weight: normal;");
var isTimeColCtrl = new ToolButtonBuilder<ToggleButton>()
.setText("")
.setTooltip("Extract timestamp from this column")
.setStyleClass("dialog-button")
.setAction(event -> {
var btn = (ToggleButton) event.getSource();
if (btn.getUserData() instanceof ColumnPosition pos) {
this.timeColumnTextField.setValueFactory(new ColumnPositionFactory(-1, 999999, pos.index()));
handleOnRunTest(null);
}
})
.setIconStyleClass("time-icon", "small-icon")
.build(ToggleButton::new);
isTimeColCtrl.setUserData(new ColumnPosition(i));
isTimeColCtrl.setSelected(i == tsColumn);
col.setGraphic(isTimeColCtrl);
col.setSortable(false);
col.setReorderable(false);
var cellFactory = new AlignedTableCellFactory<ParsedEvent, String>();
cellFactory.setAlignment(alignment);
col.setCellFactory(cellFactory);
colMap.put(col, Integer.toString(i));
return col;
}
@Override
protected void resetTest() {
super.resetTest();
this.testResultTable.getItems().clear();
this.testResultTable.getColumns().clear();
testTabPane.getSelectionModel().select(inputTab);
}
@Override
protected Optional<CsvParsingProfile> updateProfile(String profileName, String profileId, Map<NamedCaptureGroup, String> groups, String lineExpression) {
List<String> errors = new ArrayList<>();
if (this.lineTemplateExpression.getText().isBlank()) {
TextFieldValidator.fail(delimiterTextField, true);
errors.add("Timestamp pattern cannot be empty");
}
if (this.delimiterTextField.getText().isEmpty()) {
TextFieldValidator.fail(delimiterTextField, true);
errors.add("Delimiting character for CSV parsing cannot be empty");
}
if (this.quoteCharacterTextField.getText().isBlank()) {
TextFieldValidator.fail(quoteCharacterTextField, true);
errors.add("Quote character for CSV parsing cannot be empty");
}
try {
var bld = new Locale.Builder();
bld.setLanguageTag(parsingLocaleTextField.getText());
var parsingLocale = bld.build();
this.numberFormat = NumberFormat.getNumberInstance(parsingLocale);
} catch (IllformedLocaleException e) {
errors.add("The locale for number parsing is invalid: " + e.getMessage());
}
if (errors.size() > 0) {
notifyError(String.join("\n", errors));
return Optional.empty();
}
return Optional.of(new CustomCsvParsingProfile(profileName,
profileId,
groups,
lineExpression,
this.delimiterTextField.getText(),
StringUtils.stringToEscapeSequence(this.quoteCharacterTextField.getText()).charAt(0),
this.timeColumnTextField.getValue().index(),
new int[0],
this.readColumnNameCheckBox.isSelected(),
Locale.forLanguageTag(parsingLocaleTextField.getText()),
this.trimCellsCheckbox.isSelected()));
}
private String formatToDouble(String value) {
if (value != null) {
try {
return numberFormat.format(numberFormat.parse(value));
} catch (Exception e) {
// Do nothing
}
}
return "NaN";
}
}
binjr/binjr-adapter-csv/src/main/java/eu/binjr/sources/csv/data/parsers/CustomCsvParsingProfile.java
+111
View File
@@ -0,0 +1,111 @@
/*
* Copyright 2022 Frederic Thevenet
*
* Licensed under the Apache License, Version 2.0 (the "License");
* you may not use this file except in compliance with the License.
* You may obtain a copy of the License at
*
* http://www.apache.org/licenses/LICENSE-2.0
*
* Unless required by applicable law or agreed to in writing, software
* distributed under the License is distributed on an "AS IS" BASIS,
* WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
* See the License for the specific language governing permissions and
* limitations under the License.
*/
package eu.binjr.sources.csv.data.parsers;
import com.google.gson.annotations.JsonAdapter;
import eu.binjr.common.json.adapters.LocaleJsonAdapter;
import eu.binjr.core.data.indexes.parser.capture.NamedCaptureGroup;
import eu.binjr.core.data.indexes.parser.profile.CustomParsingProfile;
import java.util.HashMap;
import java.util.Locale;
import java.util.Map;
import java.util.UUID;
public class CustomCsvParsingProfile extends CustomParsingProfile implements CsvParsingProfile {
private final String delimiter;
private final char quoteCharacter;
private final int timestampColumn;
private final int[] excludedColumns;
private final boolean readColumnNames;
@JsonAdapter(LocaleJsonAdapter.class)
private final Locale formattingLocale;
private final boolean trimCellValues;
public CustomCsvParsingProfile() {
this("", UUID.randomUUID().toString(), new HashMap<>(), "", ",", '"', 0, new int[0], true, Locale.getDefault(), false);
}
public static CsvParsingProfile of(CsvParsingProfile parsingProfile) {
return new CustomCsvParsingProfile(parsingProfile.getProfileName(),
parsingProfile.getProfileId(),
parsingProfile.getCaptureGroups(),
parsingProfile.getLineTemplateExpression(),
parsingProfile.getDelimiter(),
parsingProfile.getQuoteCharacter(),
parsingProfile.getTimestampColumn(),
parsingProfile.getExcludedColumns(),
parsingProfile.isReadColumnNames(),
parsingProfile.getNumberFormattingLocale(),
parsingProfile.isTrimCellValues());
}
public CustomCsvParsingProfile(String profileName,
String profileId,
Map<NamedCaptureGroup, String> captureGroups,
String lineTemplateExpression,
String delimiter,
char quoteCharacter, int timestampColumn,
int[] excludedColumns, boolean readColumnNames,
Locale formattingLocale, boolean trimCellValues) {
super(profileName, profileId, captureGroups, lineTemplateExpression);
this.delimiter = delimiter;
this.quoteCharacter = quoteCharacter;
this.timestampColumn = timestampColumn;
this.excludedColumns = excludedColumns;
this.readColumnNames = readColumnNames;
this.formattingLocale = formattingLocale;
this.trimCellValues = trimCellValues;
}
@Override
public String getDelimiter() {
return this.delimiter;
}
@Override
public int getTimestampColumn() {
return this.timestampColumn;
}
@Override
public int[] getExcludedColumns() {
return this.excludedColumns;
}
@Override
public boolean isReadColumnNames() {
return readColumnNames;
}
@Override
public Locale getNumberFormattingLocale() {
return formattingLocale;
}
@Override
public char getQuoteCharacter() {
return quoteCharacter;
}
@Override
public boolean isTrimCellValues() {
return trimCellValues;
}
}
binjr/binjr-adapter-csv/src/main/resources/META-INF/services/eu.binjr.core.data.adapters.DataAdapterInfo
+18
View File
@@ -0,0 +1,18 @@
#
# Copyright 2017-2018 Frederic Thevenet
#
# Licensed under the Apache License, Version 2.0 (the "License");
# you may not use this file except in compliance with the License.
# You may obtain a copy of the License at
#
# http://www.apache.org/licenses/LICENSE-2.0
#
# Unless required by applicable law or agreed to in writing, software
# distributed under the License is distributed on an "AS IS" BASIS,
# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
# See the License for the specific language governing permissions and
# limitations under the License.
#
# Csv file Data adapter service implementation
eu.binjr.sources.csv.adapters.CsvFileDataAdapterInfo
binjr/binjr-adapter-csv/src/main/resources/eu/binjr/views/CsvParsingProfileDialogView.fxml
+464
View File
@@ -0,0 +1,464 @@
<?xml version="1.0" encoding="UTF-8"?>
<!--
~ Copyright 2022 Frederic Thevenet
~
~ Licensed under the Apache License, Version 2.0 (the "License");
~ you may not use this file except in compliance with the License.
~ You may obtain a copy of the License at
~
~ http://www.apache.org/licenses/LICENSE-2.0
~
~ Unless required by applicable law or agreed to in writing, software
~ distributed under the License is distributed on an "AS IS" BASIS,
~ WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
~ See the License for the specific language governing permissions and
~ limitations under the License.
-->
<?import java.lang.*?>
<?import javafx.geometry.*?>
<?import javafx.scene.control.*?>
<?import javafx.scene.control.cell.*?>
<?import javafx.scene.layout.*?>
<?import javafx.scene.text.*?>
<?import org.fxmisc.flowless.*?>
<?import org.fxmisc.richtext.*?>
<?import eu.binjr.common.javafx.controls.LabelWithInlineHelp?>
<DialogPane fx:id="dialogPane" styleClass="skinnable-pane-border, tool-dialog-window"
xmlns="http://javafx.com/javafx/17.0.2-ea" xmlns:fx="http://javafx.com/fxml/1">
<content>
<AnchorPane fx:id="root" prefHeight="777.0" prefWidth="853.0">
<children>
<VBox fx:id="expressions" spacing="10.0"
AnchorPane.bottomAnchor="4.0"
AnchorPane.leftAnchor="4.0"
AnchorPane.rightAnchor="4.0"
AnchorPane.topAnchor="40.0">
<TitledPane fx:id="setupTitledPane" animated="false" maxWidth="Infinity"
maxHeight="Infinity"
contentDisplay="GRAPHIC_ONLY"
VBox.vgrow="SOMETIMES">
<graphic>
<LabelWithInlineHelp text="Setup"
inlineHelp="Setup parsing rules&#xD;&#xD;Note: The settings below are greyed out when a built-in profile is selected. If you need to customize a built-in profile, click on &quot;Duplicate profile&quot; to create a copy that you can modify."
alignment="CENTER_RIGHT"/>
</graphic>
<VBox fx:id="setupPane" spacing="10.0">
<padding>
<Insets top="0" right="0" bottom="0" left="0"/>
</padding>
<HBox alignment="CENTER_LEFT">
<Region HBox.hgrow="ALWAYS"/>
</HBox>
<TableView fx:id="captureGroupTable" prefHeight="220.0" prefWidth="762.0"
styleClass="search-field-outer">
<columns>
<TableColumn fx:id="nameColumn" prefWidth="220.0" sortable="false"
text="Group Name">
<cellValueFactory>
<PropertyValueFactory property="name"/>
</cellValueFactory>
</TableColumn>
<TableColumn fx:id="expressionColumn" minWidth="0.0" prefWidth="550.0"
sortable="false" text="Capture Expression">
<cellValueFactory>
<PropertyValueFactory property="expression"/>
</cellValueFactory>
</TableColumn>
<TableColumn editable="false" maxWidth="56" minWidth="56" prefWidth="56.0"
resizable="false" sortable="false">
<graphic>
<HBox>
<Button fx:id="addGroupButton" cache="true"
contentDisplay="GRAPHIC_ONLY" minHeight="-Infinity"
minWidth="-Infinity" mnemonicParsing="false"
onAction="#handleOnAddGroup" prefHeight="25.0"
prefWidth="25.0"
styleClass="dialog-button" text="Add">
<graphic>
<Region>
<styleClass>
<String fx:value="icon-container"/>
<String fx:value="plus-icon"/>
<String fx:value="medium-icon"/>
</styleClass>
</Region>
</graphic>
<tooltip>
<Tooltip showDelay="500ms" text="Add new capture group"/>
</tooltip>
</Button>
<Button fx:id="deleteGroupButton" contentDisplay="GRAPHIC_ONLY"
maxHeight="-Infinity" maxWidth="-Infinity"
minHeight="-Infinity"
minWidth="-Infinity" mnemonicParsing="false"
onAction="#handleOnDeleteGroup" prefHeight="25.0"
prefWidth="25.0" styleClass="dialog-button" text="Delete">
<graphic>
<Region>
<styleClass>
<String fx:value="icon-container"/>
<String fx:value="minus-icon"/>
<String fx:value="medium-icon"/>
</styleClass>
</Region>
</graphic>
<tooltip>
<Tooltip showDelay="500ms" text="Delete capture group"/>
</tooltip>
</Button>
</HBox>
</graphic>
</TableColumn>
</columns>
</TableView>
<HBox alignment="CENTER_LEFT" spacing="10.0">
<children>
<LabelWithInlineHelp minWidth="115.0" text="Timestamp pattern" inlineHelp="The regular expression pattern be used to parse timestamps.
&#xD;Use the capture groups defined in the table above to identify the individual components of a timestamp event (e.g. day, hour, second, etc...).
&#xD;Use Java regular expression to model how these components can be recognized in the particular syntax of a CSV file."
alignment="CENTER_RIGHT"/>
<CodeArea fx:id="lineTemplateExpression" maxHeight="-Infinity"
maxWidth="1.7976931348623157E308" minHeight="26.0"
prefHeight="26.0"
styleClass="search-field-outer" HBox.hgrow="ALWAYS">
<padding>
<Insets top="2.0" right="2.0" bottom="2.0" left="2.0"/>
</padding>
</CodeArea>
</children>
</HBox>
<HBox alignment="CENTER_LEFT" spacing="10.0">
<children>
<LabelWithInlineHelp minWidth="115.0" text="Get timestamps from"
inlineHelp="Indicates in which column of the CSV file to look for timestamps data.&#xD;If your CSV file does not contain any timestamps info, you may select &quot;line number&quot;, in which case &#xD;binjr will infer a timestamp for each line, starting at the temporal reference point and &#xD;increasing by 1 second for every new line."
alignment="CENTER_RIGHT"/>
<Spinner fx:id="timeColumnTextField" editable="false"
minHeight="-Infinity" prefHeight="24.0" prefWidth="120.0"/>
<Separator orientation="VERTICAL" prefHeight="20.0"/>
<CheckBox fx:id="readColumnNameCheckBox"
text="Use values from first line as column names"/>
<CheckBox fx:id="trimCellsCheckbox"
text="Trim cell values "/>
</children>
</HBox>
<HBox alignment="CENTER_LEFT" spacing="10.0">
<children>
<LabelWithInlineHelp minWidth="115.0" text="Delimiting character"
inlineHelp="The character used to delimit the columns of the CSV file."
alignment="CENTER_RIGHT"/>
<TextField fx:id="delimiterTextField" maxWidth="35" minHeight="-Infinity"
prefHeight="24.0" prefWidth="35.0"/>
<Separator orientation="VERTICAL" prefHeight="20.0"/>
<LabelWithInlineHelp text="Quote character"
inlineHelp="The character used to encapsulate the content of a column so that it is not split even if it contains instances of the delimiting character."
alignment="CENTER_RIGHT"/>
<TextField fx:id="quoteCharacterTextField" maxWidth="35" minHeight="-Infinity"
prefHeight="24.0" prefWidth="35.0"/>
<Separator orientation="VERTICAL" prefHeight="20.0"/>
<LabelWithInlineHelp text="Locale for parsing numbers"
inlineHelp="Indicates which regional settings should be applied when parsing column content as numbers."
alignment="CENTER_RIGHT"/>
<TextField fx:id="parsingLocaleTextField" maxWidth="120.0" prefHeight="24.0"/>
</children>
</HBox>
</VBox>
</TitledPane>
<TitledPane fx:id="testTitledPane" animated="false" maxWidth="Infinity"
maxHeight="Infinity"
contentDisplay="GRAPHIC_ONLY"
VBox.vgrow="ALWAYS">
<graphic>
<LabelWithInlineHelp text="Test"
inlineHelp="Test data extraction&#xD;&#xD;
You can enter text (or paste from clipboard/load it from a file) in the &quot;input&quot; tab below to&#xD;
and see the results of your parsing rules in the &quot;result&quot; tab.&#xD;&#xD;
TIP: Clicking on the clock shaped icon next to a column's header acts as a shortcut to indicate&#xD;
that this column should used to extract timestamps from."
alignment="CENTER_RIGHT"/>
</graphic>
<VBox fx:id="testPane" spacing="10.0">
<padding>
<Insets top="0" right="0" bottom="0" left="0"/>
</padding>
<HBox alignment="CENTER">
<Button fx:id="runTestButton" contentDisplay="LEFT" maxHeight="-Infinity"
maxWidth="-Infinity"
minHeight="-Infinity" minWidth="-Infinity" mnemonicParsing="false"
onAction="#handleOnRunTest" text="Test data extraction">
<HBox.margin>
<Insets bottom="2.0" top="6.0"/>
</HBox.margin>
<graphic>
<Region>
<styleClass>
<String fx:value="icon-container"/>
<String fx:value="test-icon"/>
<String fx:value="medium-icon"/>
</styleClass>
<padding>
<Insets right="25.0"/>
</padding>
</Region>
</graphic>
<padding>
<Insets bottom="5.0" left="40.0" right="45.0" top="5.0"/>
</padding>
<tooltip>
<Tooltip showDelay="500ms" text="Test data extraction"/>
</tooltip>
</Button>
</HBox>
<Label fx:id="notificationLabel" managed="false" maxHeight="1.7976931348623157E308"
maxWidth="1.7976931348623157E308" minHeight="-Infinity"
styleClass="notification-info"
text="" visible="false" wrapText="true" VBox.vgrow="NEVER">
<padding>
<Insets bottom="4.0" left="4.0" right="4.0" top="4.0"/>
</padding>
</Label>
<HBox styleClass="search-field-outer" VBox.vgrow="ALWAYS">
<children>
<TabPane fx:id="testTabPane" maxHeight="Infinity" maxWidth="Infinity" side="TOP"
HBox.hgrow="ALWAYS">
<Tab fx:id="inputTab" closable="false" text="Input">
<graphic>
<Region prefWidth="20">
<styleClass>
<String fx:value="icon-container"/>
<String fx:value="edit-icon"/>
<String fx:value="small-icon"/>
</styleClass>
</Region>
</graphic>
<VirtualizedScrollPane>
<content>
<CodeArea fx:id="testArea" maxHeight="1.7976931348623157E308"
maxWidth="1.7976931348623157E308" prefHeight="201.0"
prefWidth="795.0" HBox.hgrow="ALWAYS"/>
</content>
</VirtualizedScrollPane>
</Tab>
<Tab fx:id="resultTab" closable="false" text="Results">
<graphic>
<Region prefWidth="20">
<styleClass>
<String fx:value="icon-container"/>
<String fx:value="eye-icon"/>
<String fx:value="small-icon"/>
</styleClass>
</Region>
</graphic>
<TableView fx:id="testResultTable" prefHeight="200.0" prefWidth="200.0"
HBox.hgrow="ALWAYS"/>
</Tab>
</TabPane>
<VBox>
<children>
<Button fx:id="clearTestAreaButton" cache="true"
contentDisplay="GRAPHIC_ONLY"
minHeight="-Infinity" minWidth="-Infinity" mnemonicParsing="false"
onAction="#handleOnClearTestArea" prefHeight="30.0" prefWidth="30.0"
styleClass="dialog-button" text="Clear">
<graphic>
<Region>
<styleClass>
<String fx:value="icon-container"/>
<String fx:value="trash-icon"/>
<String fx:value="medium-icon"/>
</styleClass>
</Region>
</graphic>
<tooltip>
<Tooltip showDelay="500ms" text="Clear test area"/>
</tooltip>
</Button>
<Button fx:id="copyTestAreaButton" cache="true"
contentDisplay="GRAPHIC_ONLY"
minHeight="-Infinity" minWidth="-Infinity" mnemonicParsing="false"
onAction="#handleOnCopyTestArea" prefHeight="30.0" prefWidth="30.0"
styleClass="dialog-button" text="Copy">
<graphic>
<Region>
<styleClass>
<String fx:value="icon-container"/>
<String fx:value="copy-icon"/>
<String fx:value="medium-icon"/>
</styleClass>
</Region>
</graphic>
<tooltip>
<Tooltip showDelay="500ms" text="Copy test area"/>
</tooltip>
</Button>
<Button fx:id="pasteTestAreaButton" contentDisplay="GRAPHIC_ONLY"
maxHeight="-Infinity" maxWidth="-Infinity" minHeight="-Infinity"
minWidth="-Infinity" mnemonicParsing="false"
onAction="#handleOnPasteToTestArea" prefHeight="30.0"
prefWidth="30.0"
styleClass="dialog-button" text="Paste">
<graphic>
<Region>
<styleClass>
<String fx:value="icon-container"/>
<String fx:value="clipboard-icon"/>
<String fx:value="medium-icon"/>
</styleClass>
</Region>
</graphic>
<tooltip>
<Tooltip showDelay="500ms" text="Paste to test area"/>
</tooltip>
</Button>
<Button fx:id="openFileButton" contentDisplay="GRAPHIC_ONLY"
maxHeight="-Infinity" maxWidth="-Infinity" minHeight="-Infinity"
minWidth="-Infinity" mnemonicParsing="false"
onAction="#handleOnOpenFileToTestArea" prefHeight="30.0"
prefWidth="30.0"
styleClass="dialog-button" text="Open File">
<graphic>
<Region>
<styleClass>
<String fx:value="icon-container"/>
<String fx:value="fileOpen-icon"/>
<String fx:value="medium-icon"/>
</styleClass>
</Region>
</graphic>
<tooltip>
<Tooltip showDelay="500ms"
text="Preview the first few lines of a file into test area"/>
</tooltip>
</Button>
<Region maxHeight="1.7976931348623157E308" VBox.vgrow="ALWAYS"/>
</children>
</VBox>
</children>
</HBox>
</VBox>
</TitledPane>
</VBox>
<HBox alignment="CENTER_LEFT" spacing="2.0"
AnchorPane.leftAnchor="4.0"
AnchorPane.rightAnchor="4.0"
AnchorPane.topAnchor="4.0">
<children>
<HBox maxHeight="1.7976931348623157E308" maxWidth="1.7976931348623157E308"
styleClass="search-field-outer" HBox.hgrow="ALWAYS">
<Region fx:id="builtinIcon" prefHeight="12" prefWidth="12" minWidth="12">
<styleClass>
<String fx:value="icon-container"/>
<String fx:value="padlock-icon"/>
<String fx:value="medium-icon"/>
</styleClass>
<HBox.margin>
<Insets left="6.0" right="0.0"/>
</HBox.margin>
</Region>
<ComboBox fx:id="profileComboBox" editable="true" maxHeight="1.7976931348623157E308"
maxWidth="1.7976931348623157E308" styleClass="search-field-inner"
HBox.hgrow="ALWAYS">
</ComboBox>
<padding>
<Insets left="1.0" right="1.0"/>
</padding>
</HBox>
<Button fx:id="addProfileButton" cache="true" contentDisplay="GRAPHIC_ONLY"
minHeight="-Infinity" minWidth="-Infinity" mnemonicParsing="false"
onAction="#handleOnAddProfile" prefHeight="30.0" prefWidth="30.0"
styleClass="dialog-button" text="Add">
<graphic>
<Region>
<styleClass>
<String fx:value="icon-container"/>
<String fx:value="plus-icon"/>
</styleClass>
</Region>
</graphic>
<tooltip>
<Tooltip showDelay="500ms" text="Create a new profile"/>
</tooltip>
</Button>
<Button fx:id="deleteProfileButton" contentDisplay="GRAPHIC_ONLY" maxHeight="-Infinity"
maxWidth="-Infinity" minHeight="-Infinity" minWidth="-Infinity" mnemonicParsing="false"
onAction="#handleOnDeleteProfile" prefHeight="30.0" prefWidth="30.0"
styleClass="dialog-button" text="delete" HBox.hgrow="NEVER">
<graphic>
<Region>
<styleClass>
<String fx:value="icon-container"/>
<String fx:value="minus-icon"/>
</styleClass>
</Region>
</graphic>
<tooltip>
<Tooltip showDelay="500ms" text="Delete profile"/>
</tooltip>
<font>
<Font size="16.0"/>
</font>
</Button>
<Button fx:id="cloneProfileButton" cache="true" contentDisplay="GRAPHIC_ONLY"
minHeight="-Infinity" minWidth="-Infinity" mnemonicParsing="false"
onAction="#handleOnCloneProfile" prefHeight="30.0" prefWidth="30.0"
styleClass="dialog-button" text="Duplicate">
<graphic>
<Region>
<styleClass>
<String fx:value="icon-container"/>
<String fx:value="copy-icon"/>
</styleClass>
</Region>
</graphic>
<tooltip>
<Tooltip showDelay="500ms" text="Duplicate profile"/>
</tooltip>
</Button>
<Button fx:id="importProfileButton" cache="true" contentDisplay="GRAPHIC_ONLY"
minHeight="-Infinity" minWidth="-Infinity" mnemonicParsing="false"
onAction="#handleOnImportProfile" prefHeight="30.0" prefWidth="30.0"
styleClass="dialog-button" text="Import" HBox.hgrow="NEVER">
<graphic>
<Region>
<styleClass>
<String fx:value="icon-container"/>
<String fx:value="upload-icon"/>
</styleClass>
</Region>
</graphic>
<tooltip>
<Tooltip showDelay="500ms" text="Import profiles"/>
</tooltip>
</Button>
<Button fx:id="exportProfileButton" contentDisplay="GRAPHIC_ONLY" maxHeight="-Infinity"
maxWidth="-Infinity" minHeight="-Infinity" minWidth="-Infinity" mnemonicParsing="false"
onAction="#handleOnExportProfile" prefHeight="30.0" prefWidth="30.0"
styleClass="dialog-button" text="Export" HBox.hgrow="NEVER">
<graphic>
<Region>
<styleClass>
<String fx:value="icon-container"/>
<String fx:value="download-icon"/>
</styleClass>
</Region>
</graphic>
<tooltip>
<Tooltip showDelay="500ms" text="Export profiles"/>
</tooltip>
</Button>
</children>
</HBox>
</children>
</AnchorPane>
</content>
<buttonTypes>
<ButtonType fx:constant="CANCEL"/>
<ButtonType fx:constant="OK"/>
</buttonTypes>
</DialogPane>
binjr/binjr-adapter-jfr/README.md
+5
View File
@@ -0,0 +1,5 @@
# binjr-adapter-jfr
[![Maven Central](https://img.shields.io/maven-central/v/eu.binjr/binjr-adapter-jfr.svg?label=Maven%20Central&style=flat-square)](https://search.maven.org/search?q=g:%22eu.binjr%22%20AND%20a:%22binjr-adapter-jfr%22)
This module implements a DataAdapter capable of consuming data from JDK Flight Recorder files.
binjr/binjr-adapter-jfr/build.gradle
+31
View File
@@ -0,0 +1,31 @@
/*
* Copyright 2023 Frederic Thevenet
*
* Licensed under the Apache License, Version 2.0 (the "License");
* you may not use this file except in compliance with the License.
* You may obtain a copy of the License at
*
* http://www.apache.org/licenses/LICENSE-2.0
*
* Unless required by applicable law or agreed to in writing, software
* distributed under the License is distributed on an "AS IS" BASIS,
* WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
* See the License for the specific language governing permissions and
* limitations under the License.
*/
dependencies {
compileOnly project(':binjr-core')
}
jar {
manifest {
attributes(
'Specification-Title': project.name,
'Specification-Version': project.version,
'Implementation-Title': project.name,
'Implementation-Version': project.version,
'Build-Number': BINJR_BUILD_NUMBER
)
}
}
binjr/binjr-adapter-jfr/src/main/java/eu/binjr/sources/jfr/adapters/BaseJfrDataAdapter.java
+199
View File
@@ -0,0 +1,199 @@
/*
* Copyright 2023 Frederic Thevenet
*
* Licensed under the Apache License, Version 2.0 (the "License");
* you may not use this file except in compliance with the License.
* You may obtain a copy of the License at
*
* http://www.apache.org/licenses/LICENSE-2.0
*
* Unless required by applicable law or agreed to in writing, software
* distributed under the License is distributed on an "AS IS" BASIS,
* WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
* See the License for the specific language governing permissions and
* limitations under the License.
*/
package eu.binjr.sources.jfr.adapters;
import com.google.gson.Gson;
import eu.binjr.common.function.CheckedLambdas;
import eu.binjr.common.io.FileSystemBrowser;
import eu.binjr.common.io.IOUtils;
import eu.binjr.common.javafx.controls.TimeRange;
import eu.binjr.common.logging.Logger;
import eu.binjr.core.data.adapters.BaseDataAdapter;
import eu.binjr.core.data.adapters.DataAdapter;
import eu.binjr.core.data.adapters.ReloadPolicy;
import eu.binjr.core.data.exceptions.CannotInitializeDataAdapterException;
import eu.binjr.core.data.exceptions.DataAdapterException;
import eu.binjr.core.data.exceptions.InvalidAdapterParameterException;
import eu.binjr.core.data.indexes.Index;
import eu.binjr.core.data.indexes.Indexes;
import eu.binjr.core.data.indexes.IndexingStatus;
import eu.binjr.core.data.indexes.parser.profile.BuiltInParsingProfile;
import eu.binjr.core.data.workspace.TimeSeriesInfo;
import javafx.beans.property.LongProperty;
import javafx.beans.property.Property;
import javafx.beans.property.SimpleLongProperty;
import javafx.beans.property.SimpleObjectProperty;
import org.apache.lucene.document.Field;
import org.apache.lucene.document.StoredField;
import org.apache.lucene.document.StringField;
import org.apache.lucene.facet.FacetField;
import java.io.IOException;
import java.lang.annotation.Retention;
import java.nio.charset.Charset;
import java.nio.file.Path;
import java.nio.file.Paths;
import java.time.ZoneId;
import java.util.*;
import java.util.concurrent.atomic.AtomicBoolean;
import java.util.stream.Collectors;
import static eu.binjr.core.data.indexes.parser.capture.CaptureGroup.SEVERITY;
/**
* A {@link DataAdapter} implementation to retrieve data from a JDK Flight Recorder file.
*
* @author Frederic Thevenet
*/
public abstract class BaseJfrDataAdapter<T> extends BaseDataAdapter<T> {
private static final Logger logger = Logger.create(BaseJfrDataAdapter.class);
protected static final Gson gson = new Gson();
protected static final Property<IndexingStatus> INDEXING_OK = new SimpleObjectProperty<>(IndexingStatus.OK);
protected static final String ZONE_ID = "zoneId";
protected static final String ENCODING = "encoding";
protected static final String PARSING_PROFILE = "parsingProfile";
protected static final String PATH = "jfrPath";
protected JfrEventFormat eventFormat;
protected Path jfrFilePath;
protected ZoneId zoneId;
protected String encoding;
protected Index index;
protected FileSystemBrowser fileBrowser;
public BaseJfrDataAdapter(Path jfrPath, ZoneId zoneId) throws DataAdapterException {
super();
initParams(zoneId, jfrPath, "utf-8");
}
@Override
public String getEncoding() {
return encoding;
}
@Override
public ZoneId getTimeZoneId() {
return zoneId;
}
@Override
public Map<String, String> getParams() {
Map<String, String> params = new HashMap<>();
params.put(ZONE_ID, zoneId.toString());
params.put(ENCODING, encoding);
params.put(PATH, jfrFilePath.toString());
return params;
}
@Override
public void loadParams(Map<String, String> params) throws DataAdapterException {
if (params == null) {
throw new InvalidAdapterParameterException("Could not find parameter list for adapter " + getSourceName());
}
initParams(validateParameter(params, ZONE_ID,
s -> {
if (s == null) {
throw new InvalidAdapterParameterException("Parameter '" + ZONE_ID + "' is missing in adapter " + getSourceName());
}
return ZoneId.of(s);
}),
Paths.get(validateParameterNullity(params, PATH)),
validateParameterNullity(params, ENCODING));
}
private void initParams(ZoneId zoneId,
Path jfrPath,
String encoding) {
this.zoneId = zoneId;
this.jfrFilePath = jfrPath;
this.encoding = encoding;
this.eventFormat = new JfrEventFormat(zoneId, Charset.forName(encoding));
}
@Override
public void onStart() throws DataAdapterException {
super.onStart();
try {
this.fileBrowser = FileSystemBrowser.of(jfrFilePath.getParent());
this.index = Indexes.LOG_FILES.acquire();
} catch (IOException e) {
throw new CannotInitializeDataAdapterException("An error occurred during the data adapter initialization", e);
}
}
@Override
public void close() {
try {
Indexes.LOG_FILES.release();
} catch (Exception e) {
logger.error("An error occurred while releasing index " + Indexes.LOG_FILES.name() + ": " + e.getMessage());
logger.debug("Stack Trace:", e);
}
IOUtils.close(fileBrowser);
super.close();
}
public synchronized void ensureIndexed(Set<String> sources, ReloadPolicy reloadPolicy) throws IOException {
if (reloadPolicy == ReloadPolicy.ALL) {
sources.forEach(index.getIndexedFiles()::remove);
}
final LongProperty charRead = new SimpleLongProperty(0);
var isCommitNecessary = false;
var filterMap = new HashMap<Path, HashSet<String>>();
for (var binding : sources) {
if (!index.getIndexedFiles().containsKey(binding)) {
var a = binding.split("\\|");
var filePath = Path.of(a[0].replace(BuiltInParsingProfile.NONE.getProfileId() + "/", ""));
var eventType = a[1];
filterMap.computeIfAbsent(filePath, p -> new HashSet<>()).add(eventType);
isCommitNecessary = true;
index.getIndexedFiles().put(binding, IndexingStatus.OK);
}
}
for (Map.Entry<Path, HashSet<String>> entry : filterMap.entrySet()) {
Path path = entry.getKey();
HashSet<String> strings = entry.getValue();
index.add(path.toString(),
new JfrRecordingFilter(path, strings),
false,
eventFormat,
(doc, event) -> {
// Add number fields
event.getNumberFields().forEach((key, value) -> doc.add(new StoredField(key, value.doubleValue())));
// Set HAS_NUM field
doc.add(new StringField(JfrEventFormat.HAS_NUM_FIELDS, event.getNumberFields().size() > 0 ? "true" : "false", Field.Store.NO));
// Add event categories as severity
String severity = event.getTextField(JfrEventFormat.CATEGORIES) == null ? "JFR" :
event.getTextField(JfrEventFormat.CATEGORIES);//.toLowerCase();
doc.add(new FacetField(SEVERITY, severity));
doc.add(new StoredField(SEVERITY, severity));
return doc;
},
charRead,
INDEXING_OK,
(rootPath, parsedEvent) -> BuiltInParsingProfile.NONE.getProfileId() + "/" + rootPath + "|" + parsedEvent.getTextField(JfrEventFormat.EVENT_TYPE_NAME),
(source) -> source.eventTypes().stream().map(type -> BuiltInParsingProfile.NONE.getProfileId() + "/" + source.recordingPath().toString() + "|" + type).toList());
}
if (isCommitNecessary) {
index.commitIndexAndTaxonomy();
}
}
}
binjr/binjr-adapter-jfr/src/main/java/eu/binjr/sources/jfr/adapters/JfrAdapterPreferences.java
+84
View File
@@ -0,0 +1,84 @@
/*
* Copyright 2023 Frederic Thevenet
*
* Licensed under the Apache License, Version 2.0 (the "License");
* you may not use this file except in compliance with the License.
* You may obtain a copy of the License at
*
* http://www.apache.org/licenses/LICENSE-2.0
*
* Unless required by applicable law or agreed to in writing, software
* distributed under the License is distributed on an "AS IS" BASIS,
* WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
* See the License for the specific language governing permissions and
* limitations under the License.
*/
package eu.binjr.sources.jfr.adapters;
import com.google.gson.Gson;
import eu.binjr.common.preferences.ObservablePreference;
import eu.binjr.core.data.adapters.DataAdapter;
import eu.binjr.core.data.adapters.DataAdapterPreferences;
import java.util.Set;
/**
* Defines the preferences associated with the Text files adapter.
*/
public class JfrAdapterPreferences extends DataAdapterPreferences {
private static final Gson gson = new Gson();
/**
* The default text panel font size preference.
*/
public ObservablePreference<Number> defaultTextViewFontSize = integerPreference("defaultTextViewFontSize", 10);
/**
* The filters used when scanning folders in the source filesystem.
*/
public ObservablePreference<String[]> folderFilters = objectPreference(String[].class,
"folderFilters",
new String[]{"*"},
gson::toJson,
s -> gson.fromJson(s, String[].class));
/**
* The filters used to prune file extensions to scan in the source filesystem.
*/
public ObservablePreference<String[]> fileExtensionFilters = objectPreference(String[].class,
"fileExtensionFilters",
new String[]{".jfr"},
gson::toJson,
s -> gson.fromJson(s, String[].class));
/**
* A list of value types for the event payload that should be included
*/
public ObservablePreference<String[]> includedEventsDataTypes = objectPreference(String[].class,
"includedEventsDataTypes",
new String[]{"short", "int", "long", "float", "double"},
gson::toJson,
s -> gson.fromJson(s, String[].class));
/**
* A list of names of events that should be excluded
*/
public ObservablePreference<String[]> excludedEventsNames = objectPreference(String[].class,
"excludedEventsByName",
new String[]{"gcId", "javaThreadId", "osThreadId", "modifiers"},
gson::toJson,
s -> gson.fromJson(s, String[].class));
/**
* Initialize a new instance of the {@link JfrAdapterPreferences} class associated to
* a {@link DataAdapter} instance.
*
* @param dataAdapterClass the associated {@link DataAdapter}
*/
public JfrAdapterPreferences(Class<? extends DataAdapter<?>> dataAdapterClass) {
super(dataAdapterClass);
}
}
binjr/binjr-adapter-jfr/src/main/java/eu/binjr/sources/jfr/adapters/JfrDataAdapter.java
+160
View File
@@ -0,0 +1,160 @@
/*
* Copyright 2023 Frederic Thevenet
*
* Licensed under the Apache License, Version 2.0 (the "License");
* you may not use this file except in compliance with the License.
* You may obtain a copy of the License at
*
* http://www.apache.org/licenses/LICENSE-2.0
*
* Unless required by applicable law or agreed to in writing, software
* distributed under the License is distributed on an "AS IS" BASIS,
* WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
* See the License for the specific language governing permissions and
* limitations under the License.
*/
package eu.binjr.sources.jfr.adapters;
import eu.binjr.common.javafx.controls.TimeRange;
import eu.binjr.common.javafx.controls.TreeViewUtils;
import eu.binjr.common.logging.Logger;
import eu.binjr.core.data.adapters.*;
import eu.binjr.core.data.exceptions.DataAdapterException;
import eu.binjr.core.data.indexes.IndexingStatus;
import eu.binjr.core.data.indexes.SearchHit;
import eu.binjr.core.data.indexes.parser.profile.BuiltInParsingProfile;
import eu.binjr.core.data.timeseries.TimeSeriesProcessor;
import eu.binjr.core.data.workspace.TimeSeriesInfo;
import javafx.beans.property.DoubleProperty;
import javafx.beans.property.Property;
import jdk.jfr.EventType;
import jdk.jfr.consumer.RecordingFile;
import org.eclipse.fx.ui.controls.tree.FilterableTreeItem;
import java.io.IOException;
import java.nio.file.Path;
import java.time.Instant;
import java.time.ZoneId;
import java.util.HashMap;
import java.util.List;
import java.util.Map;
import java.util.Set;
import java.util.stream.Collectors;
/**
* A {@link DataAdapter} implementation to retrieve data from a JDK Flight Recorder file.
*
* @author Frederic Thevenet
*/
public class JfrDataAdapter extends BaseJfrDataAdapter<SearchHit> implements ProgressAdapter<SearchHit> {
private static final Logger logger = Logger.create(JfrDataAdapter.class);
/**
* Initializes a new instance of the {@link JfrDataAdapter} class with a set of default values.
*
* @throws DataAdapterException if the {@link DataAdapter} could not be initializes.
*/
public JfrDataAdapter() throws DataAdapterException {
super(Path.of(""), ZoneId.systemDefault());
}
/**
* Initializes a new instance of the {@link JfrDataAdapter} class with the provided parameters.
*
* @param jfrPath the path to the JFR file.
* @param zoneId the time zone to used.
* @throws DataAdapterException if the {@link DataAdapter} could not be initialized.
*/
public JfrDataAdapter(Path jfrPath,
ZoneId zoneId)
throws DataAdapterException {
super(jfrPath, zoneId);
}
@Override
public TimeRange getInitialTimeRange(String path, List<TimeSeriesInfo<SearchHit>> seriesInfo) throws DataAdapterException {
try {
ensureIndexed(seriesInfo.stream().map(info -> BuiltInParsingProfile.NONE.getProfileId() + "/" + info.getBinding().getPath()).collect(Collectors.toSet()), ReloadPolicy.UNLOADED);
return index.getTimeRangeBoundaries(seriesInfo.stream().map(ts -> BuiltInParsingProfile.NONE.getProfileId() + "/" + ts.getBinding().getPath()).toList(), getTimeZoneId());
} catch (IOException e) {
throw new DataAdapterException("Error retrieving initial time range", e);
}
}
@Override
public FilterableTreeItem<SourceBinding> getBindingTree() throws DataAdapterException {
String rootPath = jfrFilePath.toString() + "|";
FilterableTreeItem<SourceBinding> tree = new FilterableTreeItem<>(new LogFilesBinding.Builder()
.withLabel(getSourceName())
.withPath(jfrFilePath.toString() + "|")
.withAdapter(this)
.build());
try (var recordingFile = new RecordingFile(jfrFilePath)) {
for (EventType eventType : recordingFile.readEventTypes()) {
var branch = tree;
for (var cat : eventType.getCategoryNames()) {
var pos = TreeViewUtils.findFirstInTree(tree, item -> item.getValue().getLabel().equals(cat));
if (pos.isEmpty()) {
var node = new FilterableTreeItem<>((SourceBinding) new LogFilesBinding.Builder()
.withLabel(cat)
.withParent(branch.getValue())
.withPath(branch.getValue().getPath() + "/" + cat)
.withAdapter(this)
.build());
branch.getInternalChildren().add(node);
branch = node;
} else {
branch = (FilterableTreeItem<SourceBinding>) pos.get();
}
}
var leaf = new FilterableTreeItem<>((SourceBinding) new LogFilesBinding.Builder()
.withLabel(eventType.getLabel())
.withLegend(eventType.getLabel())
.withPath(rootPath + eventType.getName())
.withParent(branch.getValue())
.withAdapter(this)
.build());
branch.getInternalChildren().add(leaf);
}
} catch (IOException e) {
throw new DataAdapterException("Error while attempting to read JFR recording: " + e.getMessage(), e);
}
return tree;
}
@Deprecated
@Override
public Map<TimeSeriesInfo<SearchHit>, TimeSeriesProcessor<SearchHit>> fetchData(String path,
Instant begin,
Instant end,
List<TimeSeriesInfo<SearchHit>> seriesInfo,
boolean bypassCache) throws DataAdapterException {
return loadSeries(path, seriesInfo, bypassCache ? ReloadPolicy.ALL : ReloadPolicy.UNLOADED, null, INDEXING_OK);
}
@Override
public Map<TimeSeriesInfo<SearchHit>, TimeSeriesProcessor<SearchHit>> loadSeries(String path,
List<TimeSeriesInfo<SearchHit>> seriesInfo,
ReloadPolicy reloadPolicy,
DoubleProperty progress,
Property<IndexingStatus> indexingStatus) throws DataAdapterException {
Map<TimeSeriesInfo<SearchHit>, TimeSeriesProcessor<SearchHit>> data = new HashMap<>();
try {
ensureIndexed(seriesInfo.stream().map(info -> BuiltInParsingProfile.NONE.getProfileId() + "/" + info.getBinding().getPath()).collect(Collectors.toSet()), ReloadPolicy.UNLOADED);
} catch (Exception e) {
throw new DataAdapterException("Error fetching logs from " + path, e);
}
return data;
}
@Override
public String getSourceName() {
return new StringBuilder("[JFR: Events] ")
.append(jfrFilePath != null ? jfrFilePath.getFileName() : "???")
.toString();
}
}
binjr/binjr-adapter-jfr/src/main/java/eu/binjr/sources/jfr/adapters/JfrDataAdapterDialog.java
+143
View File
@@ -0,0 +1,143 @@
/*
* Copyright 2023 Frederic Thevenet
*
* Licensed under the Apache License, Version 2.0 (the "License");
* you may not use this file except in compliance with the License.
* You may obtain a copy of the License at
*
* http://www.apache.org/licenses/LICENSE-2.0
*
* Unless required by applicable law or agreed to in writing, software
* distributed under the License is distributed on an "AS IS" BASIS,
* WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
* See the License for the specific language governing permissions and
* limitations under the License.
*/
package eu.binjr.sources.jfr.adapters;
import com.google.gson.Gson;
import eu.binjr.common.javafx.controls.NodeUtils;
import eu.binjr.common.logging.Logger;
import eu.binjr.core.data.adapters.DataAdapter;
import eu.binjr.core.data.adapters.DataAdapterFactory;
import eu.binjr.core.data.exceptions.CannotInitializeDataAdapterException;
import eu.binjr.core.data.exceptions.DataAdapterException;
import eu.binjr.core.data.exceptions.NoAdapterFoundException;
import eu.binjr.core.dialogs.DataAdapterDialog;
import eu.binjr.core.dialogs.Dialogs;
import eu.binjr.sources.jfr.adapters.charts.JfrChartsDataAdapter;
import eu.binjr.sources.jfr.adapters.charts.JfrChartsDataAdapterInfo;
import javafx.scene.Node;
import javafx.stage.FileChooser;
import java.io.File;
import java.nio.file.Files;
import java.nio.file.Path;
import java.nio.file.Paths;
import java.time.ZoneId;
import java.util.Collection;
import java.util.List;
/**
* An implementation of the {@link DataAdapterDialog} class that presents a dialog box to retrieve the parameters specific {@link JfrDataAdapterDialog}
*
* @author Frederic Thevenet
*/
public class JfrDataAdapterDialog extends DataAdapterDialog<Path> {
private static final Logger logger = Logger.create(JfrDataAdapterDialog.class);
// private final TextField extensionFiltersTextField;
private final JfrAdapterPreferences prefs;
private static final Gson gson = new Gson();
/**
* Initializes a new instance of the {@link JfrDataAdapterDialog} class.
*
* @param owner the owner window for the dialog
* @throws NoAdapterFoundException if no adapter could be found to get preferences for.
*/
public JfrDataAdapterDialog(Node owner) throws NoAdapterFoundException {
super(owner, Mode.PATH, "mostRecentJfrFiles", false);
this.prefs = (JfrAdapterPreferences) DataAdapterFactory.getInstance().getAdapterPreferences(JfrDataAdapter.class.getName());
setDialogHeaderText("Add a JFR File");
/* extensionFiltersTextField = new TextField(gson.toJson(prefs.fileExtensionFilters.get()));
var label = new Label("Extensions:");
GridPane.setConstraints(label, 0, 1, 1, 1, HPos.LEFT, VPos.CENTER, Priority.ALWAYS, Priority.ALWAYS, new Insets(4, 0, 4, 0));
GridPane.setConstraints(extensionFiltersTextField, 1, 1, 1, 1, HPos.LEFT, VPos.CENTER, Priority.ALWAYS, Priority.ALWAYS, new Insets(4, 0, 4, 0));
getParamsGridPane().getChildren().addAll(label, extensionFiltersTextField);*/
}
@Override
protected File displayFileChooser(Node owner) {
try {
/* ContextMenu sourceMenu = new ContextMenu();
MenuItem fileMenuItem = new MenuItem("JFR file");
fileMenuItem.setOnAction(eventHandler -> {
FileChooser fileChooser = new FileChooser();
fileChooser.setTitle("Open JFR File");
fileChooser.getExtensionFilters().add(new FileChooser.ExtensionFilter("JFR files", "*.jfr"));
fileChooser.getExtensionFilters().add(new FileChooser.ExtensionFilter("All files", "*.*", "*"));
Dialogs.getInitialDir(getMostRecentList()).ifPresent(fileChooser::setInitialDirectory);
File selectedFile = fileChooser.showOpenDialog(NodeUtils.getStage(owner));
if (selectedFile != null) {
setSourceUri(selectedFile.getPath());
}
});
sourceMenu.getItems().add(fileMenuItem);
MenuItem menuItem = new MenuItem("Zip file");
menuItem.setOnAction(eventHandler -> {
FileChooser fileChooser = new FileChooser();
fileChooser.setTitle("Open Zip Archive");
fileChooser.getExtensionFilters().add(new FileChooser.ExtensionFilter("Zip archive", "*.zip"));
fileChooser.getExtensionFilters().add(new FileChooser.ExtensionFilter("All files", "*.*", "*"));
Dialogs.getInitialDir(getMostRecentList()).ifPresent(fileChooser::setInitialDirectory);
File selectedFile = fileChooser.showOpenDialog(NodeUtils.getStage(owner));
if (selectedFile != null) {
setSourceUri(selectedFile.getPath());
}
});
sourceMenu.getItems().add(menuItem);
MenuItem folderMenuItem = new MenuItem("Folder");
folderMenuItem.setOnAction(eventHandler -> {
DirectoryChooser dirChooser = new DirectoryChooser();
dirChooser.setTitle("Open Folder");
Dialogs.getInitialDir(getMostRecentList()).ifPresent(dirChooser::setInitialDirectory);
File selectedFile = dirChooser.showDialog(NodeUtils.getStage(owner));
if (selectedFile != null) {
setSourceUri(selectedFile.getPath());
}
});
sourceMenu.getItems().add(folderMenuItem);
sourceMenu.show(owner, Side.RIGHT, 0, 0);*/
FileChooser fileChooser = new FileChooser();
fileChooser.setTitle("Open JFR File");
fileChooser.getExtensionFilters().add(new FileChooser.ExtensionFilter("JFR files", "*.jfr"));
fileChooser.getExtensionFilters().add(new FileChooser.ExtensionFilter("All files", "*.*", "*"));
Dialogs.getInitialDir(getMostRecentList()).ifPresent(fileChooser::setInitialDirectory);
File selectedFile = fileChooser.showOpenDialog(NodeUtils.getStage(owner));
if (selectedFile != null) {
return selectedFile;
}
} catch (Exception e) {
Dialogs.notifyException("Error while displaying file chooser: " + e.getMessage(), e, owner);
}
return null;
}
@Override
protected Collection<DataAdapter> getDataAdapters() throws DataAdapterException {
Path path = Paths.get(getSourceUri());
if (!Files.exists(path)) {
throw new CannotInitializeDataAdapterException("Cannot find " + getSourceUri());
}
if (!path.isAbsolute()) {
throw new CannotInitializeDataAdapterException("The provided path is not valid.");
}
getMostRecentList().push(path);
// prefs.fileExtensionFilters.set(gson.fromJson(extensionFiltersTextField.getText(), String[].class));
return List.of(new JfrDataAdapter(path, ZoneId.of(getSourceTimezone())),
new JfrChartsDataAdapter(path, ZoneId.of(getSourceTimezone())));
}
}
binjr/binjr-adapter-jfr/src/main/java/eu/binjr/sources/jfr/adapters/JfrDataAdapterInfo.java
+57
View File
@@ -0,0 +1,57 @@
/*
* Copyright 2023 Frederic Thevenet
*
* Licensed under the Apache License, Version 2.0 (the "License");
* you may not use this file except in compliance with the License.
* You may obtain a copy of the License at
*
* http://www.apache.org/licenses/LICENSE-2.0
*
* Unless required by applicable law or agreed to in writing, software
* distributed under the License is distributed on an "AS IS" BASIS,
* WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
* See the License for the specific language governing permissions and
* limitations under the License.
*/
package eu.binjr.sources.jfr.adapters;
import eu.binjr.core.data.adapters.AdapterMetadata;
import eu.binjr.core.data.adapters.BaseDataAdapterInfo;
import eu.binjr.core.data.adapters.SourceLocality;
import eu.binjr.core.data.adapters.VisualizationType;
import eu.binjr.core.data.exceptions.CannotInitializeDataAdapterException;
import eu.binjr.core.preferences.AppEnvironment;
/**
* Defines the metadata associated with JfrDataAdapter.
*
* @author Frederic Thevenet
*/
@AdapterMetadata(
name = "JFR",
description = "JDK Flight Recorder Files Data Adapter (Events view)",
copyright = AppEnvironment.COPYRIGHT_NOTICE,
license = AppEnvironment.LICENSE,
siteUrl = AppEnvironment.HTTP_WWW_BINJR_EU,
adapterClass = JfrDataAdapter.class,
dialogClass = JfrDataAdapterDialog.class,
preferencesClass = JfrAdapterPreferences.class,
sourceLocality = SourceLocality.LOCAL,
apiLevel = AppEnvironment.PLUGIN_API_LEVEL,
visualizationType = VisualizationType.EVENTS
)
public class JfrDataAdapterInfo extends BaseDataAdapterInfo {
/**
* Initialises a new instance of the {@link JfrDataAdapterInfo} class.
*
* @throws CannotInitializeDataAdapterException if the adapter's initialization failed
*/
public JfrDataAdapterInfo() throws CannotInitializeDataAdapterException {
super(JfrDataAdapterInfo.class);
}
}
binjr/binjr-adapter-jfr/src/main/java/eu/binjr/sources/jfr/adapters/JfrEventFormat.java
+101
View File
@@ -0,0 +1,101 @@
/*
* Copyright 2023 Frederic Thevenet
*
* Licensed under the Apache License, Version 2.0 (the "License");
* you may not use this file except in compliance with the License.
* You may obtain a copy of the License at
*
* http://www.apache.org/licenses/LICENSE-2.0
*
* Unless required by applicable law or agreed to in writing, software
* distributed under the License is distributed on an "AS IS" BASIS,
* WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
* See the License for the specific language governing permissions and
* limitations under the License.
*/
package eu.binjr.sources.jfr.adapters;
import eu.binjr.common.logging.Logger;
import eu.binjr.core.data.adapters.DataAdapterFactory;
import eu.binjr.core.data.exceptions.NoAdapterFoundException;
import eu.binjr.core.data.indexes.parser.EventFormat;
import eu.binjr.core.data.indexes.parser.EventParser;
import eu.binjr.core.data.indexes.parser.profile.BuiltInParsingProfile;
import eu.binjr.core.data.indexes.parser.profile.ParsingProfile;
import jdk.jfr.MemoryAddress;
import jdk.jfr.Timestamp;
import jdk.jfr.ValueDescriptor;
import java.nio.charset.Charset;
import java.time.ZoneId;
import java.util.Arrays;
import java.util.Objects;
public class JfrEventFormat implements EventFormat<JfrRecordingFilter> {
public static final String EVENT_TYPE_NAME = "eventTypeName";
public static final String JDK_CPULOAD = "jdk.CPULoad";
public static final String JVM_SYSTEM = "jvmSystem";
public static final String JVM_USER = "jvmUser";
public static final String MACHINE_TOTAL = "machineTotal";
private static final Logger logger = Logger.create(JfrEventParser.class);
public static final String CATEGORIES = "categories";
public static final String HAS_NUM_FIELDS = "hasNumFields";
public static final String GCREF_TYPE_FIELD = "type";
public static final String JDK_GCREFERENCE_STATISTICS = "jdk.GCReferenceStatistics";
public static final String GCREF_COUNT_FIELD = "count";
public static final String JDK_TYPES_THREAD_GROUP = "jdk.types.ThreadGroup";
public static final String GCREF_FINAL_REFERENCE = "Final reference";
public static final String GCREF_SOFT_REFERENCE = "Soft reference";
public static final String GCREF_WEAK_REFERENCE = "Weak reference";
public static final String GCREF_PHANTOM_REFERENCE = "Phantom reference";
public static final String GCREF_TOTAL_COUNT = "Total Count";
public static final String JDK_TYPES_STACK_TRACE = "jdk.types.StackTrace";
public static final String JDK_GARBAGE_COLLECTION = "jdk.GarbageCollection";
private final ZoneId zoneId;
private final Charset encoding;
private static final JfrAdapterPreferences adapterPrefs;
static {
try {
adapterPrefs = (JfrAdapterPreferences) DataAdapterFactory.getInstance().getAdapterPreferences(JfrDataAdapter.class.getName());
} catch (NoAdapterFoundException e) {
throw new RuntimeException(e);
}
}
public JfrEventFormat(ZoneId zoneId, Charset encoding) {
this.zoneId = zoneId;
this.encoding = encoding;
}
@Override
public ParsingProfile getProfile() {
return BuiltInParsingProfile.NONE;
}
@Override
public EventParser parse(JfrRecordingFilter source) {
return new JfrEventParser(this, source);
}
@Override
public Charset getEncoding() {
return encoding;
}
@Override
public ZoneId getZoneId() {
return zoneId;
}
public static boolean includeField(ValueDescriptor field) {
return field.getAnnotation(Timestamp.class) == null &&
field.getAnnotation(MemoryAddress.class) ==null &&
Arrays.stream(adapterPrefs.includedEventsDataTypes.get()).anyMatch(s -> Objects.equals(s, field.getTypeName())) &&
Arrays.stream(adapterPrefs.excludedEventsNames.get()).noneMatch(s -> Objects.equals(s, field.getName()));
}
}
binjr/binjr-adapter-jfr/src/main/java/eu/binjr/sources/jfr/adapters/JfrEventParser.java
+139
View File
@@ -0,0 +1,139 @@
/*
* Copyright 2023 Frederic Thevenet
*
* Licensed under the Apache License, Version 2.0 (the "License");
* you may not use this file except in compliance with the License.
* You may obtain a copy of the License at
*
* http://www.apache.org/licenses/LICENSE-2.0
*
* Unless required by applicable law or agreed to in writing, software
* distributed under the License is distributed on an "AS IS" BASIS,
* WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
* See the License for the specific language governing permissions and
* limitations under the License.
*/
package eu.binjr.sources.jfr.adapters;
import eu.binjr.common.logging.Logger;
import eu.binjr.common.logging.Profiler;
import eu.binjr.core.data.indexes.parser.EventParser;
import eu.binjr.core.data.indexes.parser.ParsedEvent;
import javafx.beans.property.LongProperty;
import javafx.beans.property.SimpleLongProperty;
import jdk.jfr.consumer.RecordedEvent;
import jdk.jfr.consumer.RecordedObject;
import jdk.jfr.consumer.RecordingFile;
import java.io.IOException;
import java.time.ZonedDateTime;
import java.util.Iterator;
import java.util.LinkedHashMap;
import java.util.Map;
import java.util.concurrent.atomic.AtomicLong;
public class JfrEventParser implements EventParser {
private static final Logger logger = Logger.create(JfrEventParser.class);
// private final BufferedReader reader;
private final AtomicLong sequence;
private final JfrEventFormat format;
private final JfrEventIterator eventIterator;
private final RecordingFile recordingFile;
private final LongProperty progress = new SimpleLongProperty(0);
private final JfrRecordingFilter eventTypeFilter;
JfrEventParser(JfrEventFormat format, JfrRecordingFilter filter) {
this.sequence = new AtomicLong(0);
this.format = format;
try (var p = Profiler.start("Initialize JFR Recording file", logger::perf)) {
this.eventTypeFilter = filter;
this.recordingFile = new RecordingFile(filter.recordingPath());
this.eventIterator = new JfrEventIterator();
} catch (IOException e) {
throw new RuntimeException(e);
}
}
@Override
public void close() throws IOException {
recordingFile.close();
}
@Override
public LongProperty progressIndicator() {
return progress;
}
@Override
public Iterator<ParsedEvent> iterator() {
return eventIterator;
}
public class JfrEventIterator implements Iterator<ParsedEvent> {
@Override
public ParsedEvent next() {
ParsedEvent event = null;
while (event == null && recordingFile.hasMoreEvents()) {
event = readNextJrfEvent();
}
return event;
}
private ParsedEvent readNextJrfEvent() {
RecordedEvent jfrEvent = null;
try {
jfrEvent = recordingFile.readEvent();
if (!eventTypeFilter.eventTypes().contains(jfrEvent.getEventType().getName())) {
return null;
}
ZonedDateTime timestamp = ZonedDateTime.ofInstant(jfrEvent.getStartTime(), format.getZoneId());
Map<String, Number> numFields = new LinkedHashMap<>();
var categories = jfrEvent.getEventType().getCategoryNames();
String catFacet = categories.size() > 1 ? categories.get(1) : categories.get(0);
switch (jfrEvent.getEventType().getName()) {
case JfrEventFormat.JDK_GCREFERENCE_STATISTICS ->
numFields.put(String.join(" ", jfrEvent.getValue(JfrEventFormat.GCREF_TYPE_FIELD),
JfrEventFormat.GCREF_TOTAL_COUNT), jfrEvent.getValue(JfrEventFormat.GCREF_COUNT_FIELD));
default -> addField("", jfrEvent, numFields);
}
return new ParsedEvent(
sequence.incrementAndGet(),
timestamp,
jfrEvent.toString(),
Map.of(JfrEventFormat.CATEGORIES, catFacet, JfrEventFormat.EVENT_TYPE_NAME, jfrEvent.getEventType().getName()),
numFields);
} catch (Exception e) {
logger.error("Error parsing JFR event [" + (jfrEvent == null ? "unknown" : jfrEvent.getEventType().getName()) + "]: " + e.getMessage());
logger.debug("Stack trace", e);
return null;
}
}
private void addField(String parentLabel, RecordedObject jfrEvent, Map<String, Number> numFields) {
for (var field : jfrEvent.getFields()) {
if (JfrEventFormat.includeField(field)) {
numFields.put(String.join(" ", parentLabel, field.getLabel()).trim(), jfrEvent.getValue(field.getName()));
}
if (!field.getFields().isEmpty() && jfrEvent.getValue(field.getName()) instanceof RecordedObject nestedEvent) {
addField(field.getLabel(), nestedEvent, numFields);
}
}
}
@Override
public boolean hasNext() {
return recordingFile.hasMoreEvents();
}
}
}
binjr/binjr-adapter-jfr/src/main/java/eu/binjr/sources/jfr/adapters/JfrRecordingFilter.java
+23
View File
@@ -0,0 +1,23 @@
/*
* Copyright 2023 Frederic Thevenet
*
* Licensed under the Apache License, Version 2.0 (the "License");
* you may not use this file except in compliance with the License.
* You may obtain a copy of the License at
*
* http://www.apache.org/licenses/LICENSE-2.0
*
* Unless required by applicable law or agreed to in writing, software
* distributed under the License is distributed on an "AS IS" BASIS,
* WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
* See the License for the specific language governing permissions and
* limitations under the License.
*/
package eu.binjr.sources.jfr.adapters;
import java.nio.file.Path;
import java.util.Set;
public record JfrRecordingFilter(Path recordingPath, Set<String> eventTypes) {
}
binjr/binjr-adapter-jfr/src/main/java/eu/binjr/sources/jfr/adapters/charts/JfrChartsDataAdapter.java
+285
View File
@@ -0,0 +1,285 @@
/*
* Copyright 2023 Frederic Thevenet
*
* Licensed under the Apache License, Version 2.0 (the "License");
* you may not use this file except in compliance with the License.
* You may obtain a copy of the License at
*
* http://www.apache.org/licenses/LICENSE-2.0
*
* Unless required by applicable law or agreed to in writing, software
* distributed under the License is distributed on an "AS IS" BASIS,
* WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
* See the License for the specific language governing permissions and
* limitations under the License.
*/
package eu.binjr.sources.jfr.adapters.charts;
import eu.binjr.common.javafx.controls.TimeRange;
import eu.binjr.common.javafx.controls.TreeViewUtils;
import eu.binjr.common.logging.Logger;
import eu.binjr.core.data.adapters.DataAdapter;
import eu.binjr.core.data.adapters.ReloadPolicy;
import eu.binjr.core.data.adapters.SourceBinding;
import eu.binjr.core.data.adapters.TimeSeriesBinding;
import eu.binjr.core.data.exceptions.DataAdapterException;
import eu.binjr.core.data.indexes.parser.profile.BuiltInParsingProfile;
import eu.binjr.core.data.timeseries.DoubleTimeSeriesProcessor;
import eu.binjr.core.data.timeseries.TimeSeriesProcessor;
import eu.binjr.core.data.workspace.ChartType;
import eu.binjr.core.data.workspace.TimeSeriesInfo;
import eu.binjr.core.data.workspace.UnitPrefixes;
import eu.binjr.sources.jfr.adapters.BaseJfrDataAdapter;
import eu.binjr.sources.jfr.adapters.JfrEventFormat;
import javafx.scene.paint.Color;
import jdk.jfr.*;
import jdk.jfr.consumer.RecordingFile;
import org.eclipse.fx.ui.controls.tree.FilterableTreeItem;
import java.io.IOException;
import java.nio.file.Path;
import java.time.Instant;
import java.time.ZoneId;
import java.util.HashMap;
import java.util.List;
import java.util.Map;
import java.util.stream.Collectors;
/**
* A {@link DataAdapter} implementation to retrieve data from a JDK Flight Recorder file.
*
* @author Frederic Thevenet
*/
public class JfrChartsDataAdapter extends BaseJfrDataAdapter<Double> {
private static final Logger logger = Logger.create(JfrChartsDataAdapter.class);
public static final int RECURSE_MAX_DEPTH = 10;
/**
* Initializes a new instance of the {@link JfrChartsDataAdapter} class with a set of default values.
*
* @throws DataAdapterException if the {@link DataAdapter} could not be initializes.
*/
public JfrChartsDataAdapter() throws DataAdapterException {
super(Path.of(""), ZoneId.systemDefault());
}
/**
* Initializes a new instance of the {@link JfrChartsDataAdapter} class with the provided parameters.
*
* @param jfrPath the path to the JFR file.
* @param zoneId the time zone to used.
* @throws DataAdapterException if the {@link DataAdapter} could not be initialized.
*/
public JfrChartsDataAdapter(Path jfrPath,
ZoneId zoneId)
throws DataAdapterException {
super(jfrPath, zoneId);
}
@Override
public TimeRange getInitialTimeRange(String path, List<TimeSeriesInfo<Double>> seriesInfo) throws DataAdapterException {
try {
ensureIndexed(seriesInfo.stream().map(info -> info.getBinding().getPath()).collect(Collectors.toSet()), ReloadPolicy.UNLOADED);
return index.getTimeRangeBoundaries(seriesInfo.stream().map(ts -> ts.getBinding().getPath()).toList(), getTimeZoneId());
} catch (IOException e) {
throw new DataAdapterException("Error retrieving initial time range", e);
}
}
@Override
public FilterableTreeItem<SourceBinding> getBindingTree() throws DataAdapterException {
String rootPath = BuiltInParsingProfile.NONE.getProfileId() + "/" + jfrFilePath.toString() + "|";
FilterableTreeItem<SourceBinding> tree = new FilterableTreeItem<>(new TimeSeriesBinding.Builder()
.withLabel(getSourceName())
.withPath(rootPath)
.withAdapter(this)
.build());
try (var recordingFile = new RecordingFile(jfrFilePath)) {
for (EventType eventType : recordingFile.readEventTypes()) {
var branch = tree;
for (var cat : eventType.getCategoryNames()) {
var pos = TreeViewUtils.findFirstInTree(tree, item -> item.getValue().getLabel().equals(cat));
if (pos.isEmpty()) {
var node = new FilterableTreeItem<>((SourceBinding) new TimeSeriesBinding.Builder()
.withLabel(cat)
.withParent(branch.getValue())
.withPath(branch.getValue().getPath() + "/" + cat)
.withAdapter(this)
.build());
branch.getInternalChildren().add(node);
branch = node;
} else {
branch = (FilterableTreeItem<SourceBinding>) pos.get();
}
}
var leaf = new FilterableTreeItem<>((SourceBinding) new TimeSeriesBinding.Builder()
.withLabel(eventType.getLabel())
.withLegend(eventType.getLabel())
.withPath(rootPath + eventType.getName())
.withParent(branch.getValue())
.withGraphType(ChartType.LINE)
.withAdapter(this)
.build());
branch.getInternalChildren().add(leaf);
switch (eventType.getName()) {
case JfrEventFormat.JDK_GCREFERENCE_STATISTICS -> {
addField(JfrEventFormat.GCREF_FINAL_REFERENCE, eventType.getField(JfrEventFormat.GCREF_COUNT_FIELD), leaf, 1);
addField(JfrEventFormat.GCREF_SOFT_REFERENCE, eventType.getField(JfrEventFormat.GCREF_COUNT_FIELD), leaf, 1);
addField(JfrEventFormat.GCREF_WEAK_REFERENCE, eventType.getField(JfrEventFormat.GCREF_COUNT_FIELD), leaf, 1);
addField(JfrEventFormat.GCREF_PHANTOM_REFERENCE, eventType.getField(JfrEventFormat.GCREF_COUNT_FIELD), leaf, 1);
}
case JfrEventFormat.JDK_CPULOAD ->{
var jvmBranch =new FilterableTreeItem<>((SourceBinding) new TimeSeriesBinding.Builder()
.withLabel("JVM")
.withPath(leaf.getValue().getPath())
.withParent(leaf.getValue())
.withUnitName("%")
.withPrefix(UnitPrefixes.PERCENTAGE)
.withGraphType(ChartType.STACKED)
.withAdapter(this)
.build());
leaf.getInternalChildren().add(jvmBranch);
jvmBranch.getInternalChildren().add(new FilterableTreeItem<>(new TimeSeriesBinding.Builder()
.withLabel(eventType.getField(JfrEventFormat.JVM_SYSTEM).getLabel())
.withPath(jvmBranch.getValue().getPath())
.withParent(jvmBranch.getValue())
.withUnitName("%")
.withPrefix(UnitPrefixes.PERCENTAGE)
.withColor(Color.RED)
.withGraphType(ChartType.LINE)
.withAdapter(this)
.build()));
jvmBranch.getInternalChildren().add(new FilterableTreeItem<>(new TimeSeriesBinding.Builder()
.withLabel(eventType.getField(JfrEventFormat.JVM_USER).getLabel())
.withPath(jvmBranch.getValue().getPath())
.withParent(jvmBranch.getValue())
.withUnitName("%")
.withPrefix(UnitPrefixes.PERCENTAGE)
.withColor(Color.DARKGREEN)
.withGraphType(ChartType.LINE)
.withAdapter(this)
.build()));
var machineBranch =new FilterableTreeItem<>((SourceBinding) new TimeSeriesBinding.Builder()
.withLabel("Machine")
.withPath(leaf.getValue().getPath())
.withParent(leaf.getValue())
.withUnitName("%")
.withPrefix(UnitPrefixes.PERCENTAGE)
.withGraphType(ChartType.LINE)
.withAdapter(this)
.build());
leaf.getInternalChildren().add(machineBranch);
machineBranch.getInternalChildren().add(new FilterableTreeItem<>(new TimeSeriesBinding.Builder()
.withLabel(eventType.getField(JfrEventFormat.MACHINE_TOTAL).getLabel())
.withPath(machineBranch.getValue().getPath())
.withParent(machineBranch.getValue())
.withUnitName("%")
.withPrefix(UnitPrefixes.PERCENTAGE)
.withGraphType(ChartType.LINE)
.withAdapter(this)
.build()));
}
default -> {
for (var field : eventType.getFields()) {
addField("", field, leaf, 1);
}
}
}
}
} catch (Exception e) {
throw new DataAdapterException("Error while attempting to read JFR recording: " + e.getMessage(), e);
}
return tree;
}
private void addField(String parentName, ValueDescriptor field, FilterableTreeItem<SourceBinding> leaf, int depth) {
if (JfrEventFormat.includeField(field)) {
var unit = extractKnownUnit(field);
leaf.getInternalChildren().add(new FilterableTreeItem<>(new TimeSeriesBinding.Builder()
.withLabel(String.join(" ", parentName, field.getLabel()).trim())
.withPath(leaf.getValue().getPath())
.withParent(leaf.getValue())
.withUnitName(unit.name())
.withPrefix(unit.prefix())
.withGraphType(ChartType.LINE)
.withAdapter(this)
.build()));
}
for (var nestedField : field.getFields()) {
if (!field.getTypeName().equals(JfrEventFormat.JDK_TYPES_THREAD_GROUP) &&
!field.getTypeName().equals(JfrEventFormat.JDK_TYPES_STACK_TRACE) &&
depth <= RECURSE_MAX_DEPTH) {
addField(field.getLabel(), nestedField, leaf, depth + 1);
}
}
}
@Override
public Map<TimeSeriesInfo<Double>, TimeSeriesProcessor<Double>> fetchData(String path,
Instant begin,
Instant end,
List<TimeSeriesInfo<Double>> seriesInfo,
boolean bypassCache) throws DataAdapterException {
try {
ensureIndexed(seriesInfo.stream().map(info -> info.getBinding().getPath()).collect(Collectors.toSet()), ReloadPolicy.UNLOADED);
Map<TimeSeriesInfo<Double>, TimeSeriesProcessor<Double>> series = new HashMap<>();
for (TimeSeriesInfo<Double> info : seriesInfo) {
series.put(info, new DoubleTimeSeriesProcessor());
}
var nbHits = index.search(
begin.toEpochMilli(),
end.toEpochMilli(),
series,
getTimeZoneId(),
bypassCache);
logger.debug(() -> "Retrieved " + nbHits + " hits");
return series;
} catch (Exception e) {
throw new DataAdapterException("Error fetching data from " + path, e);
}
}
@Override
public String getSourceName() {
return new StringBuilder("[JFR: Charts] ")
.append(jfrFilePath != null ? jfrFilePath.getFileName() : "???")
.toString();
}
private Unit extractKnownUnit(ValueDescriptor field) {
var timespan = field.getAnnotation(Timespan.class);
if (timespan != null) {
return new Unit(timespan.value(), UnitPrefixes.METRIC);
}
var frequency = field.getAnnotation(Frequency.class);
if (frequency != null) {
return new Unit("Hertz", UnitPrefixes.METRIC);
}
var dataAmount = field.getAnnotation(DataAmount.class);
if (dataAmount != null) {
return new Unit(dataAmount.value(), UnitPrefixes.BINARY);
}
var percentage = field.getAnnotation(Percentage.class);
if (percentage != null) {
return new Unit("%", UnitPrefixes.NONE);
}
return new Unit("-", UnitPrefixes.NONE);
}
private record Unit(String name, UnitPrefixes prefix) {
}
}
binjr/binjr-adapter-jfr/src/main/java/eu/binjr/sources/jfr/adapters/charts/JfrChartsDataAdapterInfo.java
+61
View File
@@ -0,0 +1,61 @@
/*
* Copyright 2023 Frederic Thevenet
*
* Licensed under the Apache License, Version 2.0 (the "License");
* you may not use this file except in compliance with the License.
* You may obtain a copy of the License at
*
* http://www.apache.org/licenses/LICENSE-2.0
*
* Unless required by applicable law or agreed to in writing, software
* distributed under the License is distributed on an "AS IS" BASIS,
* WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
* See the License for the specific language governing permissions and
* limitations under the License.
*/
package eu.binjr.sources.jfr.adapters.charts;
import eu.binjr.core.data.adapters.AdapterMetadata;
import eu.binjr.core.data.adapters.BaseDataAdapterInfo;
import eu.binjr.core.data.adapters.SourceLocality;
import eu.binjr.core.data.adapters.VisualizationType;
import eu.binjr.core.data.exceptions.CannotInitializeDataAdapterException;
import eu.binjr.core.preferences.AppEnvironment;
import eu.binjr.sources.jfr.adapters.JfrDataAdapterDialog;
/**
* Defines the metadata associated with JfrDataAdapter.
*
* @author Frederic Thevenet
*/
@AdapterMetadata(
name = "JFR - Charts)",
description = "JDK Flight Recorder Charts Data Adapter",
copyright = AppEnvironment.COPYRIGHT_NOTICE,
license = AppEnvironment.LICENSE,
siteUrl = AppEnvironment.HTTP_WWW_BINJR_EU,
adapterClass = JfrChartsDataAdapter.class,
dialogClass = JfrDataAdapterDialog.class,
sourceLocality = SourceLocality.LOCAL,
apiLevel = AppEnvironment.PLUGIN_API_LEVEL,
visualizationType = VisualizationType.CHARTS
)
public class JfrChartsDataAdapterInfo extends BaseDataAdapterInfo {
/**
* Initialises a new instance of the {@link JfrChartsDataAdapterInfo} class.
*
* @throws CannotInitializeDataAdapterException if the adapter's initialization failed
*/
public JfrChartsDataAdapterInfo() throws CannotInitializeDataAdapterException {
super(JfrChartsDataAdapterInfo.class);
}
@Override
public boolean isEnabled() {
return false;
}
}
binjr/binjr-adapter-jfr/src/main/resources/META-INF/services/eu.binjr.core.data.adapters.DataAdapterInfo
+20
View File
@@ -0,0 +1,20 @@
#
# Copyright 2023 Frederic Thevenet
#
# Licensed under the Apache License, Version 2.0 (the "License");
# you may not use this file except in compliance with the License.
# You may obtain a copy of the License at
#
# http://www.apache.org/licenses/LICENSE-2.0
#
# Unless required by applicable law or agreed to in writing, software
# distributed under the License is distributed on an "AS IS" BASIS,
# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
# See the License for the specific language governing permissions and
# limitations under the License.
#
# JFR Data adapter service implementation
eu.binjr.sources.jfr.adapters.JfrDataAdapterInfo
eu.binjr.sources.jfr.adapters.charts.JfrChartsDataAdapterInfo
binjr/binjr-adapter-jrds/README.md
+15
View File
@@ -0,0 +1,15 @@
# binjr-adapter-jrds
[![Maven Central](https://img.shields.io/maven-central/v/eu.binjr/binjr-adapter-jrds.svg?label=Maven%20Central&style=flat-square)](https://search.maven.org/search?q=g:%22eu.binjr%22%20AND%20a:%22binjr-adapter-jrds%22)
This module implements a DataAdapter capable of consuming data from a [JRDS](https://github.com/fbacchella/jrds) instance.
JRDS is a monitoring and performance collection application. It already proposes a web based front-end that allow end-users
to visualize time-series as graphs which is based on [RRD4J](https://github.com/rrd4j/rrd4j), but does so by producing static images that can’t be
zoomed in or out.
Using this DataAdapter, you can connect to a JRDS server via HTTP and use the flexibility offered by binjr to
navigate through the data.
This adapter supports authentication via Kerberos (see [here](https://github.com/binjr/binjr/wiki/Troubleshooting#kerberos-authentication-issues)
if you need help getting it to work).
binjr/binjr-adapter-jrds/build.gradle
+31
View File
@@ -0,0 +1,31 @@
/*
* Copyright 2018-2021 Frederic Thevenet
*
* Licensed under the Apache License, Version 2.0 (the "License");
* you may not use this file except in compliance with the License.
* You may obtain a copy of the License at
*
* http://www.apache.org/licenses/LICENSE-2.0
*
* Unless required by applicable law or agreed to in writing, software
* distributed under the License is distributed on an "AS IS" BASIS,
* WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
* See the License for the specific language governing permissions and
* limitations under the License.
*/
dependencies {
compileOnly project(':binjr-core')
}
jar {
manifest {
attributes(
'Specification-Title': project.name,
'Specification-Version': project.version,
'Implementation-Title': project.name,
'Implementation-Version': project.version,
'Build-Number': BINJR_BUILD_NUMBER
)
}
}
binjr/binjr-adapter-jrds/src/main/java/eu/binjr/sources/jrds/adapters/Graphdesc.java
+141
View File
@@ -0,0 +1,141 @@
/*
* Copyright 2017-2020 Frederic Thevenet
*
* Licensed under the Apache License, Version 2.0 (the "License");
* you may not use this file except in compliance with the License.
* You may obtain a copy of the License at
*
* http://www.apache.org/licenses/LICENSE-2.0
*
* Unless required by applicable law or agreed to in writing, software
* distributed under the License is distributed on an "AS IS" BASIS,
* WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
* See the License for the specific language governing permissions and
* limitations under the License.
*/
package eu.binjr.sources.jrds.adapters;
import jakarta.xml.bind.annotation.*;
import java.util.List;
import java.util.stream.Collectors;
/**
* A annotated POJO class used to deserialize JRDS graph descriptor XML messages return by {@code /graphdesc?id=""} service.
*
* @author Frederic Thevenet
*/
@XmlAccessorType(XmlAccessType.FIELD)
@XmlRootElement(name = "graphdec")
class Graphdesc {
@XmlElement(name = "name")
String name;
@XmlElement(name = "graphName")
String graphName;
@XmlElement(name = "graphClass")
String graphClass;
@XmlElement(name = "graphTitle")
String graphTitle;
@XmlElementWrapper(name = "unit")
@XmlElements({
@XmlElement(name = "SI", type = JrdsMetricUnitType.class),
@XmlElement(name = "binary", type = JrdsBinaryUnitType.class)
})
List<JrdsUnitType> unit;
@XmlElement(name = "verticalLabel")
String verticalLabel;
@XmlElement(name = "upperLimit")
String upperLimit;
@XmlElement(name = "lowerLimit")
String lowerLimit;
@XmlElement(name = "logarithmic")
String logarithmic;
@XmlElement(name = "add")
List<SeriesDesc> seriesDescList;
@XmlElement(name = "tree")
List<Tree> trees;
@Override
public String toString() {
final StringBuilder sb = new StringBuilder("Graphdesc{");
sb.append("name='").append(name).append('\'');
sb.append(", graphName='").append(graphName).append('\'');
sb.append(", graphClass='").append(graphClass).append('\'');
sb.append(", graphTitle='").append(graphTitle).append('\'');
sb.append(", unit='").append(unit).append('\'');
sb.append(", verticalLabel='").append(verticalLabel).append('\'');
sb.append(", upperLimit='").append(upperLimit).append('\'');
sb.append(", lowerLimit='").append(lowerLimit).append('\'');
sb.append(", logarithmic='").append(logarithmic).append('\'');
sb.append(", seriesDescList=").append(seriesDescList.stream().map(SeriesDesc::toString).collect(Collectors.joining(";")));
sb.append(", trees=").append(trees.stream().map(Tree::toString).collect(Collectors.joining(";")));
sb.append('}');
return sb.toString();
}
@XmlAccessorType(XmlAccessType.FIELD)
@XmlRootElement(name = "add")
static class SeriesDesc {
@XmlElement(name = "name")
String name = "";
@XmlElement(name = "dsName")
String dsName = "";
@XmlElement(name = "graphType")
String graphType = "";
@XmlElement(name = "color")
String color = "";
@XmlElement(name = "legend")
String legend = "";
@XmlElement(name = "rpn")
String rpn = "";
@Override
public String toString() {
final StringBuilder sb = new StringBuilder("SeriesDesc{");
sb.append("name='").append(name).append('\'');
sb.append(", dsName='").append(dsName).append('\'');
sb.append(", graphType='").append(graphType).append('\'');
sb.append(", color='").append(color).append('\'');
sb.append(", legend='").append(legend).append('\'');
sb.append(", rpn='").append(rpn).append('\'');
sb.append('}');
return sb.toString();
}
}
@XmlAccessorType(XmlAccessType.FIELD)
@XmlRootElement(name = "tree")
static class Tree {
@XmlElement(name = "pathstring")
List<String> pathstring;
@Override
public String toString() {
final StringBuilder sb = new StringBuilder("Tree{");
sb.append("pathstring=").append(pathstring);
sb.append('}');
return sb.toString();
}
}
@XmlSeeAlso({JrdsMetricUnitType.class, JrdsBinaryUnitType.class})
public static abstract class JrdsUnitType {
}
@XmlType(name = "SI")
public static class JrdsMetricUnitType extends JrdsUnitType {
@Override
public String toString() {
return "SI";
}
}
@XmlType(name = "binary")
public static class JrdsBinaryUnitType extends JrdsUnitType {
@Override
public String toString() {
return "binary";
}
}
}
binjr/binjr-adapter-jrds/src/main/java/eu/binjr/sources/jrds/adapters/JrdsAdapterDialog.java
+80
View File
@@ -0,0 +1,80 @@
/*
* Copyright 2017-2020 Frederic Thevenet
*
* Licensed under the Apache License, Version 2.0 (the "License");
* you may not use this file except in compliance with the License.
* You may obtain a copy of the License at
*
* http://www.apache.org/licenses/LICENSE-2.0
*
* Unless required by applicable law or agreed to in writing, software
* distributed under the License is distributed on an "AS IS" BASIS,
* WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
* See the License for the specific language governing permissions and
* limitations under the License.
*/
package eu.binjr.sources.jrds.adapters;
import eu.binjr.core.data.adapters.DataAdapter;
import eu.binjr.core.data.exceptions.DataAdapterException;
import eu.binjr.core.dialogs.DataAdapterDialog;
import javafx.beans.binding.Bindings;
import javafx.geometry.HPos;
import javafx.geometry.Insets;
import javafx.geometry.VPos;
import javafx.scene.Node;
import javafx.scene.control.ChoiceBox;
import javafx.scene.control.Label;
import javafx.scene.control.TextField;
import javafx.scene.layout.GridPane;
import javafx.scene.layout.HBox;
import javafx.scene.layout.Priority;
import java.net.URI;
import java.time.ZoneId;
import java.util.Collection;
import java.util.List;
/**
* An implementation of the {@link DataAdapterDialog} class that presents a dialog box to retrieve the parameters specific {@link JrdsDataAdapter}
*
* @author Frederic Thevenet
*/
public class JrdsAdapterDialog extends DataAdapterDialog<URI> {
private final ChoiceBox<JrdsTreeViewTab> tabsChoiceBox;
private final TextField extraArgumentTextField;
/**
* Initializes a new instance of the {@link JrdsAdapterDialog} class.
*
* @param owner the owner window for the dialog
*/
public JrdsAdapterDialog(Node owner) {
super(owner, Mode.URI, "mostRecentJrdsUrls", true);
this.setDialogHeaderText("Connect to a JRDS source");
this.tabsChoiceBox = new ChoiceBox<>();
tabsChoiceBox.getItems().addAll(JrdsTreeViewTab.values());
this.extraArgumentTextField = new TextField();
HBox.setHgrow(extraArgumentTextField, Priority.ALWAYS);
HBox hBox = new HBox(tabsChoiceBox, extraArgumentTextField);
hBox.setSpacing(10);
GridPane.setConstraints(hBox, 1, 2, 1, 1, HPos.LEFT, VPos.CENTER, Priority.ALWAYS, Priority.ALWAYS, new Insets(4, 0, 4, 0));
tabsChoiceBox.getSelectionModel().select(JrdsTreeViewTab.HOSTS_TAB);
Label tabsLabel = new Label("Sorted By:");
GridPane.setConstraints(tabsLabel, 0, 2, 1, 1, HPos.LEFT, VPos.CENTER, Priority.ALWAYS, Priority.ALWAYS, new Insets(4, 0, 4, 0));
getParamsGridPane().getChildren().addAll(tabsLabel, hBox);
extraArgumentTextField.visibleProperty().bind(Bindings.createBooleanBinding(() -> this.tabsChoiceBox.valueProperty().get().getArgument() != null, this.tabsChoiceBox.valueProperty()));
}
@Override
protected Collection<DataAdapter> getDataAdapters() throws DataAdapterException {
return List.of(JrdsDataAdapter.fromUrl(
getSourceUri(),
ZoneId.of(getSourceTimezone()),
this.tabsChoiceBox.getValue(),
this.extraArgumentTextField.getText()));
}
}
binjr/binjr-adapter-jrds/src/main/java/eu/binjr/sources/jrds/adapters/JrdsBindingBuilder.java
+118
View File
@@ -0,0 +1,118 @@
/*
* Copyright 2020 Frederic Thevenet
*
* Licensed under the Apache License, Version 2.0 (the "License");
* you may not use this file except in compliance with the License.
* You may obtain a copy of the License at
*
* http://www.apache.org/licenses/LICENSE-2.0
*
* Unless required by applicable law or agreed to in writing, software
* distributed under the License is distributed on an "AS IS" BASIS,
* WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
* See the License for the specific language governing permissions and
* limitations under the License.
*/
package eu.binjr.sources.jrds.adapters;
import eu.binjr.common.logging.Logger;
import eu.binjr.core.data.adapters.TimeSeriesBinding;
import eu.binjr.core.data.workspace.ChartType;
import eu.binjr.core.data.workspace.UnitPrefixes;
import javafx.scene.paint.Color;
/**
* Factory used to build instances of {@link eu.binjr.core.data.adapters.SourceBinding} from values
* configured by the setters, for use with {@link JrdsDataAdapter}
*/
public class JrdsBindingBuilder extends TimeSeriesBinding.Builder {
private static final Logger logger = Logger.create(JrdsBindingBuilder.class);
@Override
protected JrdsBindingBuilder self() {
return this;
}
/**
* Sets the builder with all the parameters contained in a {@link Graphdesc} instance.
*
* @param graphdesc the graphdesc
* @return This builder.
*/
public JrdsBindingBuilder withGraphDesc(Graphdesc graphdesc) {
withLabel(isNullOrEmpty(graphdesc.name) ?
(isNullOrEmpty(graphdesc.graphName) ?
"???" : graphdesc.graphName) : graphdesc.name);
withGraphType(getChartType(graphdesc));
withPrefix(findPrefix(graphdesc));
withUnitName(graphdesc.verticalLabel);
return self();
}
/**
* Sets the builder with all the parameters contained in the {@link eu.binjr.sources.jrds.adapters.Graphdesc.SeriesDesc}
* at the provided index
*
* @param graphdesc the graphdesc
* @param idx the index
* @return This builder.
*/
public JrdsBindingBuilder withGraphDesc(Graphdesc graphdesc, int idx) {
Graphdesc.SeriesDesc desc = graphdesc.seriesDescList.get(idx);
withLabel(isNullOrEmpty(desc.name) ?
(isNullOrEmpty(desc.dsName) ?
(isNullOrEmpty(desc.legend) ?
"???" : desc.legend) : desc.dsName) : desc.name);
Color c = null;
try {
if (!isNullOrEmpty(desc.color)) {
c = Color.web(desc.color);
}
} catch (IllegalArgumentException e) {
logger.warn("Invalid color string for binding " + toString());
}
withColor(c);
withLegend(isNullOrEmpty(desc.legend) ?
(isNullOrEmpty(desc.name) ?
(isNullOrEmpty(desc.dsName) ?
"???" : desc.dsName) : desc.name) : desc.legend);
withGraphType(getChartType(desc));
withPrefix(findPrefix(graphdesc));
withUnitName(graphdesc.verticalLabel);
return self();
}
private UnitPrefixes findPrefix(Graphdesc graphdesc) {
if (graphdesc.unit != null && graphdesc.unit.size() > 0) {
if (graphdesc.unit.get(0) instanceof Graphdesc.JrdsMetricUnitType) {
return UnitPrefixes.METRIC;
}
if (graphdesc.unit.get(0) instanceof Graphdesc.JrdsBinaryUnitType) {
return UnitPrefixes.BINARY;
}
}
return UnitPrefixes.UNDEFINED;
}
private ChartType getChartType(Graphdesc graphdesc) {
return graphdesc.seriesDescList.stream()
.filter(desc -> !desc.graphType.equalsIgnoreCase("none") && !desc.graphType.equalsIgnoreCase("comment"))
.reduce((last, n) -> n)
.map(this::getChartType)
.orElse(ChartType.UNDEFINED);
}
private ChartType getChartType(Graphdesc.SeriesDesc desc) {
return switch (desc.graphType.toLowerCase()) {
case "area" -> ChartType.AREA;
case "line" -> ChartType.LINE;
case "stacked" -> ChartType.STACKED;
default -> ChartType.UNDEFINED;
};
}
private boolean isNullOrEmpty(String s) {
return s == null || s.trim().length() == 0;
}
}
binjr/binjr-adapter-jrds/src/main/java/eu/binjr/sources/jrds/adapters/JrdsDataAdapter.java
+404
View File
@@ -0,0 +1,404 @@
/*
* Copyright 2016-2023 Frederic Thevenet
*
* Licensed under the Apache License, Version 2.0 (the "License");
* you may not use this file except in compliance with the License.
* You may obtain a copy of the License at
*
* http://www.apache.org/licenses/LICENSE-2.0
*
* Unless required by applicable law or agreed to in writing, software
* distributed under the License is distributed on an "AS IS" BASIS,
* WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
* See the License for the specific language governing permissions and
* limitations under the License.
*/
package eu.binjr.sources.jrds.adapters;
import com.google.gson.Gson;
import com.google.gson.JsonParseException;
import eu.binjr.common.logging.Logger;
import eu.binjr.common.xml.XmlUtils;
import eu.binjr.core.data.adapters.*;
import eu.binjr.core.data.codec.csv.CsvDecoder;
import eu.binjr.core.data.exceptions.DataAdapterException;
import eu.binjr.core.data.exceptions.FetchingDataFromAdapterException;
import eu.binjr.core.data.exceptions.InvalidAdapterParameterException;
import eu.binjr.core.data.exceptions.SourceCommunicationException;
import eu.binjr.core.data.timeseries.DoubleTimeSeriesProcessor;
import eu.binjr.core.dialogs.Dialogs;
import eu.binjr.sources.jrds.adapters.json.JsonJrdsItem;
import eu.binjr.sources.jrds.adapters.json.JsonJrdsTree;
import jakarta.xml.bind.JAXB;
import jakarta.xml.bind.annotation.XmlAccessType;
import jakarta.xml.bind.annotation.XmlAccessorType;
import javafx.beans.value.ChangeListener;
import javafx.beans.value.ObservableValue;
import org.eclipse.fx.ui.controls.tree.FilterableTreeItem;
import java.io.IOException;
import java.io.InputStream;
import java.net.URI;
import java.net.URISyntaxException;
import java.net.URL;
import java.net.http.HttpResponse;
import java.time.Instant;
import java.time.ZoneId;
import java.time.ZonedDateTime;
import java.time.format.DateTimeFormatter;
import java.util.*;
import java.util.stream.Collectors;
/**
* This class provides an implementation of {@link SerializedDataAdapter} for JRDS.
*
* @author Frederic Thevenet
*/
@XmlAccessorType(XmlAccessType.FIELD)
public class JrdsDataAdapter extends HttpDataAdapter<Double> {
private static final String JRDS_FILTER = "filter";
private static final String JRDS_TREE = "tree";
private static final String ENCODING_PARAM_NAME = "encoding";
private static final String ZONE_ID_PARAM_NAME = "zoneId";
private static final String TREE_VIEW_TAB_PARAM_NAME = "treeViewTab";
private static final Logger logger = Logger.create(JrdsDataAdapter.class);
private static final char DELIMITER = ',';
private final Gson gson;
private CsvDecoder decoder;
private String filter;
private ZoneId zoneId;
private String encoding;
private JrdsTreeViewTab treeViewTab;
/**
* Initialises a new instance of the {@link JrdsDataAdapter} class.
*
* @throws DataAdapterException if an error occurs while initializing the adapter.
*/
public JrdsDataAdapter() throws DataAdapterException {
this(null, ZoneId.systemDefault(), "utf-8", JrdsTreeViewTab.HOSTS_TAB, "");
}
/**
* Initializes a new instance of the {@link JrdsDataAdapter} class.
*
* @param baseURL the URL to the JRDS webapp.
* @param zoneId the id of the time zone used to record dates.
* @param encoding the encoding used by the download servlet.
* @param treeViewTab the tab to apply.
* @param filter the filter to apply to the tree view.
* @throws DataAdapterException if an error occurs while initializing the adapter.
*/
public JrdsDataAdapter(URL baseURL, ZoneId zoneId, String encoding, JrdsTreeViewTab treeViewTab, String filter) throws DataAdapterException {
super(baseURL);
this.zoneId = zoneId;
this.encoding = encoding;
this.treeViewTab = treeViewTab;
this.filter = filter;
this.decoder = decoderFactory(zoneId);
gson = new Gson();
}
/**
* Builds a new instance of the {@link JrdsDataAdapter} class from the provided parameters.
*
* @param address the URL to the JRDS webapp.
* @param zoneId the id of the time zone used to record dates.
* @param treeViewTab the tab to apply.
* @param filter the filter to apply to the tree view.
* @return a new instance of the {@link JrdsDataAdapter} class.
* @throws DataAdapterException if an error occurs while initializing the adapter.
*/
public static JrdsDataAdapter fromUrl(String address, ZoneId zoneId, JrdsTreeViewTab treeViewTab, String filter) throws DataAdapterException {
return new JrdsDataAdapter(urlFromString(address), zoneId, "utf-8", treeViewTab, filter);
}
@Override
public FilterableTreeItem<SourceBinding> getBindingTree() throws DataAdapterException {
try {
JsonJrdsTree t = gson.fromJson(getJsonTree(treeViewTab.getCommand(), treeViewTab.getArgument(), filter), JsonJrdsTree.class);
Map<String, JsonJrdsItem> m = Arrays.stream(t.items).collect(Collectors.toMap(o -> o.id, (o -> o)));
FilterableTreeItem<SourceBinding> tree = new FilterableTreeItem<>(
new JrdsBindingBuilder()
.withLabel(getSourceName())
.withPath("/")
.withAdapter(this)
.build());
for (JsonJrdsItem branch : Arrays.stream(t.items)
.filter(jsonJrdsItem -> JRDS_TREE.equals(jsonJrdsItem.type) || JRDS_FILTER.equals(jsonJrdsItem.type))
.collect(Collectors.toList())) {
attachNode(tree, branch.id, m);
}
return tree;
} catch (JsonParseException e) {
throw new DataAdapterException("An error occurred while parsing the json response to getBindingTree request", e);
} catch (URISyntaxException e) {
throw new SourceCommunicationException("Error building URI for request", e);
}
}
@Override
protected URI craftFetchUri(String path, Instant begin, Instant end) throws DataAdapterException {
return craftRequestUri("download",
new UriParameter("id", path),
new UriParameter("begin", Long.toString(begin.toEpochMilli())),
new UriParameter("end", Long.toString(end.toEpochMilli()))
);
}
@Override
public String getSourceName() {
return new StringBuilder("[JRDS] ")
.append(getBaseAddress() != null ? getBaseAddress().getHost() : "???")
.append((getBaseAddress() != null && getBaseAddress().getPort() > 0) ? ":" + getBaseAddress().getPort() : "")
.append(" - ")
.append(treeViewTab != null ? treeViewTab : "???")
.append(filter != null ? filter : "")
.append(" (")
.append(zoneId != null ? zoneId : "???")
.append(")").toString();
}
@Override
public Map<String, String> getParams() {
Map<String, String> params = new HashMap<>(super.getParams());
params.put(ZONE_ID_PARAM_NAME, zoneId.toString());
params.put(ENCODING_PARAM_NAME, encoding);
params.put(TREE_VIEW_TAB_PARAM_NAME, treeViewTab.name());
params.put(JRDS_FILTER, this.filter);
return params;
}
@Override
public void loadParams(Map<String, String> params) throws DataAdapterException {
if (params == null) {
throw new InvalidAdapterParameterException("Could not find parameter list for adapter " + getSourceName());
}
super.loadParams(params);
encoding = validateParameterNullity(params, ENCODING_PARAM_NAME);
zoneId = validateParameter(params, ZONE_ID_PARAM_NAME,
s -> {
if (s == null) {
throw new InvalidAdapterParameterException("Parameter " + ZONE_ID_PARAM_NAME + " is missing in adapter " + getSourceName());
}
return ZoneId.of(s);
});
treeViewTab = validateParameter(params, TREE_VIEW_TAB_PARAM_NAME, s -> s == null ? JrdsTreeViewTab.valueOf(params.get(TREE_VIEW_TAB_PARAM_NAME)) : JrdsTreeViewTab.HOSTS_TAB);
this.filter = params.get(JRDS_FILTER);
this.decoder = decoderFactory(zoneId);
}
@Override
public String getEncoding() {
return encoding;
}
@Override
public ZoneId getTimeZoneId() {
return zoneId;
}
@Override
public CsvDecoder getDecoder() {
return decoder;
}
@Override
public void close() {
super.close();
}
/**
* Returns a collection of filters provided by the JRDS server.
*
* @return a collection of filters provided by the JRDS server.
* @throws DataAdapterException if an error occurs while parsing the server's response.
* @throws URISyntaxException if the generated url is not valid.
*/
public Collection<String> discoverFilters() throws DataAdapterException, URISyntaxException {
try {
JsonJrdsTree t = gson.fromJson(getJsonTree(treeViewTab.getCommand(), treeViewTab.getArgument()), JsonJrdsTree.class);
return Arrays.stream(t.items).filter(jsonJrdsItem -> JRDS_FILTER.equals(jsonJrdsItem.type)).map(i -> i.filter).collect(Collectors.toList());
} catch (JsonParseException e) {
throw new DataAdapterException("An error occurred while parsing the json response to getBindingTree request", e);
}
}
private void attachNode(FilterableTreeItem<SourceBinding> tree, String id, Map<String, JsonJrdsItem> nodes) throws DataAdapterException {
JsonJrdsItem n = nodes.get(id);
String currentPath = normalizeId(n.id);
FilterableTreeItem<SourceBinding> newBranch = new FilterableTreeItem<>(
new JrdsBindingBuilder()
.withParent(tree.getValue())
.withLabel(n.name)
.withPath(currentPath)
.withAdapter(this)
.build());
if (JRDS_FILTER.equals(n.type)) {
// add a dummy node so that the branch can be expanded
newBranch.getInternalChildren().add(new FilterableTreeItem<>(null));
// add a listener that will get the treeview filtered according to the selected filter/tag
newBranch.expandedProperty().addListener(new FilteredViewListener(n, newBranch));
} else {
if (n.children != null) {
for (JsonJrdsItem.JsonTreeRef ref : n.children) {
attachNode(newBranch, ref._reference, nodes);
}
} else {
// add a dummy node so that the branch can be expanded
newBranch.getInternalChildren().add(new FilterableTreeItem<>(null));
// add a listener so that bindings for individual datastore are added lazily to avoid
// dozens of individual call to "graphdesc" when the tree is built.
newBranch.expandedProperty().addListener(new GraphDescListener(currentPath, newBranch, tree));
}
}
tree.getInternalChildren().add(newBranch);
}
private String normalizeId(String id) {
if (id == null || id.trim().length() == 0) {
throw new IllegalArgumentException("Argument id cannot be null or blank");
}
String[] data = id.split("\\.");
return data[data.length - 1];
}
private String getJsonTree(String tabName, String argName) throws DataAdapterException, URISyntaxException {
return getJsonTree(tabName, argName, null);
}
private String getJsonTree(String tabName, String argName, String argValue) throws DataAdapterException, URISyntaxException {
List<NameValuePair> params = new ArrayList<>();
params.add(new UriParameter("tab", tabName));
if (argName != null && argValue != null && argValue.trim().length() > 0) {
params.add(new UriParameter(argName, argValue));
}
String entityString = doHttpGetJson(craftRequestUri("jsontree", params));
logger.trace(entityString);
return entityString;
}
private Graphdesc getGraphDescriptor(String id) throws DataAdapterException {
URI requestUri = craftRequestUri("graphdesc", new UriParameter("id", id));
return doHttpGet(requestUri, responseInfo -> HttpResponse.BodySubscribers.mapping(HttpResponse.BodySubscribers.ofInputStream(), body -> {
if (responseInfo.statusCode() == 404) {
// This is probably an older version of JRDS that doesn't provide the graphdesc service,
// so we're falling back to recovering the datastore name from the csv file provided by
// the download service.
logger.warn("Cannot found graphdesc service; falling back to legacy mode.");
try {
return getGraphDescriptorLegacy(id);
} catch (Exception e) {
throw new RuntimeException(e);
}
}
if (body == null) {
return null;
}
try {
return JAXB.unmarshal(XmlUtils.toNonValidatingSAXSource(body), Graphdesc.class);
} catch (Exception e) {
throw new RuntimeException("Failed to unmarshall graphdesc response", e);
}
}));
}
private Graphdesc getGraphDescriptorLegacy(String id) throws DataAdapterException {
Instant now = ZonedDateTime.now().toInstant();
try (InputStream in = fetchRawData(id, now.minusSeconds(300), now, false)) {
List<String> headers = getDecoder().getDataColumnHeaders(in);
Graphdesc desc = new Graphdesc();
desc.seriesDescList = new ArrayList<>();
for (String header : headers) {
Graphdesc.SeriesDesc d = new Graphdesc.SeriesDesc();
d.name = header;
desc.seriesDescList.add(d);
}
return desc;
} catch (IOException e) {
throw new FetchingDataFromAdapterException(e);
}
}
private CsvDecoder decoderFactory(ZoneId zoneId) {
return new CsvDecoder(getEncoding(), DELIMITER,
DoubleTimeSeriesProcessor::new,
s -> ZonedDateTime.parse(s, DateTimeFormatter.ofPattern("yyyy-MM-dd HH:mm:ss").withZone(zoneId)));
}
private class GraphDescListener implements ChangeListener<Boolean> {
private final String currentPath;
private final FilterableTreeItem<SourceBinding> newBranch;
private final FilterableTreeItem<SourceBinding> tree;
public GraphDescListener(String currentPath, FilterableTreeItem<SourceBinding> newBranch, FilterableTreeItem<SourceBinding> tree) {
this.currentPath = currentPath;
this.newBranch = newBranch;
this.tree = tree;
}
@Override
public void changed(ObservableValue<? extends Boolean> observable, Boolean oldValue, Boolean newValue) {
if (newValue) {
try {
Graphdesc graphdesc = getGraphDescriptor(currentPath);
newBranch.setValue(new JrdsBindingBuilder()
.withGraphDesc(graphdesc)
.withParent(tree.getValue())
.withLegend(newBranch.getValue().getLegend())
.withPath(currentPath)
.withAdapter(JrdsDataAdapter.this)
.build());
for (int i = 0; i < graphdesc.seriesDescList.size(); i++) {
String graphType = graphdesc.seriesDescList.get(i).graphType;
if (!"none".equalsIgnoreCase(graphType) && !"comment".equalsIgnoreCase(graphType)) {
newBranch.getInternalChildren().add(new FilterableTreeItem<>((new JrdsBindingBuilder()
.withGraphDesc(graphdesc, i)
.withParent(newBranch.getValue())
.withPath(currentPath)
.withAdapter(JrdsDataAdapter.this)
.build())));
}
}
//remove dummy node
newBranch.getInternalChildren().remove(0);
// remove the listener so it isn't executed next time node is expanded
newBranch.expandedProperty().removeListener(this);
} catch (Exception e) {
Dialogs.notifyException("Failed to retrieve graph description", e);
}
}
}
}
private class FilteredViewListener implements ChangeListener<Boolean> {
private final JsonJrdsItem n;
private final FilterableTreeItem<SourceBinding> newBranch;
public FilteredViewListener(JsonJrdsItem n, FilterableTreeItem<SourceBinding> newBranch) {
this.n = n;
this.newBranch = newBranch;
}
@Override
public void changed(ObservableValue<? extends Boolean> observable, Boolean oldValue, Boolean newValue) {
if (newValue) {
try {
JsonJrdsTree t = gson.fromJson(getJsonTree(treeViewTab.getCommand(), JRDS_FILTER, n.name), JsonJrdsTree.class);
Map<String, JsonJrdsItem> m = Arrays.stream(t.items).collect(Collectors.toMap(o -> o.id, (o -> o)));
for (JsonJrdsItem branch : Arrays.stream(t.items).filter(jsonJrdsItem -> JRDS_TREE.equals(jsonJrdsItem.type) || JRDS_FILTER.equals(jsonJrdsItem.type)).collect(Collectors.toList())) {
attachNode(newBranch, branch.id, m);
}
//remove dummy node
newBranch.getInternalChildren().remove(0);
// remove the listener so it isn't executed next time node is expanded
newBranch.expandedProperty().removeListener(this);
} catch (Exception e) {
Dialogs.notifyException("Failed to retrieve graph description", e);
}
}
}
}
}
binjr/binjr-adapter-jrds/src/main/java/eu/binjr/sources/jrds/adapters/JrdsDataAdapterInfo.java
+53
View File
@@ -0,0 +1,53 @@
/*
* Copyright 2017-2023 Frederic Thevenet
*
* Licensed under the Apache License, Version 2.0 (the "License");
* you may not use this file except in compliance with the License.
* You may obtain a copy of the License at
*
* http://www.apache.org/licenses/LICENSE-2.0
*
* Unless required by applicable law or agreed to in writing, software
* distributed under the License is distributed on an "AS IS" BASIS,
* WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
* See the License for the specific language governing permissions and
* limitations under the License.
*/
package eu.binjr.sources.jrds.adapters;
import eu.binjr.core.data.adapters.AdapterMetadata;
import eu.binjr.core.data.adapters.BaseDataAdapterInfo;
import eu.binjr.core.data.adapters.SourceLocality;
import eu.binjr.core.data.adapters.VisualizationType;
import eu.binjr.core.data.exceptions.CannotInitializeDataAdapterException;
import eu.binjr.core.preferences.AppEnvironment;
/**
* Defines the metadata associated with the JrdsDataAdapter.
*
* @author Frederic Thevenet
*/
@AdapterMetadata(
name = "JRDS",
description = "JRDS Data Adapter",
copyright = AppEnvironment.COPYRIGHT_NOTICE,
license = AppEnvironment.LICENSE,
siteUrl = AppEnvironment.HTTP_WWW_BINJR_EU,
adapterClass = JrdsDataAdapter.class,
dialogClass = JrdsAdapterDialog.class,
sourceLocality = SourceLocality.REMOTE,
apiLevel = AppEnvironment.PLUGIN_API_LEVEL,
visualizationType = VisualizationType.CHARTS
)
public class JrdsDataAdapterInfo extends BaseDataAdapterInfo {
/**
* Initialises a new instance of the {@link JrdsDataAdapterInfo} class.
*
* @throws CannotInitializeDataAdapterException if the adapter's initialization failed
*/
public JrdsDataAdapterInfo() throws CannotInitializeDataAdapterException {
super(JrdsDataAdapterInfo.class);
}
}
binjr/binjr-adapter-jrds/src/main/java/eu/binjr/sources/jrds/adapters/JrdsTreeViewTab.java
+90
View File
@@ -0,0 +1,90 @@
/*
* Copyright 2017-2020 Frederic Thevenet
*
* Licensed under the Apache License, Version 2.0 (the "License");
* you may not use this file except in compliance with the License.
* You may obtain a copy of the License at
*
* http://www.apache.org/licenses/LICENSE-2.0
*
* Unless required by applicable law or agreed to in writing, software
* distributed under the License is distributed on an "AS IS" BASIS,
* WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
* See the License for the specific language governing permissions and
* limitations under the License.
*/
package eu.binjr.sources.jrds.adapters;
/**
* An enumeration of JRDS tree tabs.
*
* @author Frederic Thevenet
*/
public enum JrdsTreeViewTab {
/**
* All Hosts
*/
HOSTS_TAB("All Hosts", "hoststab", null),
/**
* Single Host
*/
SINGLE_HOST("Host: ", "hoststab", "host"),
/**
* All Services
*/
SERVICE_TAB("All Services", "servicestab", null),
/**
* All Views
*/
VIEWS_TAB("All Views", "viewstab", null),
/**
* All Filters
*/
FILTER_TAB("All Filters", "filtertab", null),
/**
* Single Filter
*/
SINGLE_FILTER("Filter: ", "filtertab", "filter"),
/**
* All Tags
*/
TAGS_TAB("All Tags", "tagstab", null),
/**
* Single Tag
*/
SINGLE_TAG("Tag: ", "tagstab", "filter");
private final String argument;
private final String label;
private final String command;
JrdsTreeViewTab(String label, String command, String argument) {
this.label = label;
this.command = command;
this.argument = argument;
}
@Override
public String toString() {
return label;
}
/**
* Gets the command keyword associated with the enum entry
*
* @return the command keyword associated with the enum entry
*/
public String getCommand() {
return command;
}
/**
* Gets the argument associated with the enum entry
*
* @return the argument associated with the enum entry
*/
public String getArgument() {
return argument;
}
}
binjr/binjr-adapter-jrds/src/main/java/eu/binjr/sources/jrds/adapters/json/JsonJrdsItem.java
+66
View File
@@ -0,0 +1,66 @@
/*
* Copyright 2017-2018 Frederic Thevenet
*
* Licensed under the Apache License, Version 2.0 (the "License");
* you may not use this file except in compliance with the License.
* You may obtain a copy of the License at
*
* http://www.apache.org/licenses/LICENSE-2.0
*
* Unless required by applicable law or agreed to in writing, software
* distributed under the License is distributed on an "AS IS" BASIS,
* WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
* See the License for the specific language governing permissions and
* limitations under the License.
*/
package eu.binjr.sources.jrds.adapters.json;
import java.util.Arrays;
/**
* JRDS item data class
*/
public class JsonJrdsItem {
/**
* The name
*/
public String name;
/**
* The id
*/
public String id;
/**
* The type
*/
public String type;
/**
* The filter
*/
public String filter;
/**
* Children list
*/
public JsonTreeRef[] children;
/**
* TreeRef
*/
public static class JsonTreeRef {
/**
* reference
*/
public String _reference;
}
@Override
public String toString() {
return "JsonJrdsItem{" +
"name='" + name + '\'' +
", id='" + id + '\'' +
", type='" + type + '\'' +
", filter='" + filter + '\'' +
", children=" + Arrays.toString(children) +
'}';
}
}
binjr/binjr-adapter-jrds/src/main/java/eu/binjr/sources/jrds/adapters/json/JsonJrdsStatus.java
+51
View File
@@ -0,0 +1,51 @@
/*
* Copyright 2017-2018 Frederic Thevenet
*
* Licensed under the Apache License, Version 2.0 (the "License");
* you may not use this file except in compliance with the License.
* You may obtain a copy of the License at
*
* http://www.apache.org/licenses/LICENSE-2.0
*
* Unless required by applicable law or agreed to in writing, software
* distributed under the License is distributed on an "AS IS" BASIS,
* WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
* See the License for the specific language governing permissions and
* limitations under the License.
*/
package eu.binjr.sources.jrds.adapters.json;
import java.util.Arrays;
/**
* Data class for JRDS status
*/
public class JsonJrdsStatus {
/**
* Hosts
*/
public int Hosts;
/**
* Probes
*/
public int Probes;
/**
* Timers
*/
public JsonJrdsTimer[] Timers;
/**
* Generation
*/
public int Generation;
@Override
public String toString() {
return "JsonJrdsStatus{" +
"Hosts=" + Hosts +
", Probes=" + Probes +
", Timers=" + Arrays.toString(Timers) +
", Generation=" + Generation +
'}';
}
}
binjr/binjr-adapter-jrds/src/main/java/eu/binjr/sources/jrds/adapters/json/JsonJrdsTimer.java
+44
View File
@@ -0,0 +1,44 @@
/*
* Copyright 2017-2018 Frederic Thevenet
*
* Licensed under the Apache License, Version 2.0 (the "License");
* you may not use this file except in compliance with the License.
* You may obtain a copy of the License at
*
* http://www.apache.org/licenses/LICENSE-2.0
*
* Unless required by applicable law or agreed to in writing, software
* distributed under the License is distributed on an "AS IS" BASIS,
* WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
* See the License for the specific language governing permissions and
* limitations under the License.
*/
package eu.binjr.sources.jrds.adapters.json;
/**
* Data class for JRDS Timer
*/
public class JsonJrdsTimer {
/**
* Name
*/
public String Name;
/**
* Last collect
*/
public long LastCollect;
/**
* Last duration
*/
public long LastDuration;
@Override
public String toString() {
return "JsonJrdsTimer{" +
"Name='" + Name + '\'' +
", LastCollect=" + LastCollect +
", LastDuration=" + LastDuration +
'}';
}
}
binjr/binjr-adapter-jrds/src/main/java/eu/binjr/sources/jrds/adapters/json/JsonJrdsTree.java
+46
View File
@@ -0,0 +1,46 @@
/*
* Copyright 2017-2018 Frederic Thevenet
*
* Licensed under the Apache License, Version 2.0 (the "License");
* you may not use this file except in compliance with the License.
* You may obtain a copy of the License at
*
* http://www.apache.org/licenses/LICENSE-2.0
*
* Unless required by applicable law or agreed to in writing, software
* distributed under the License is distributed on an "AS IS" BASIS,
* WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
* See the License for the specific language governing permissions and
* limitations under the License.
*/
package eu.binjr.sources.jrds.adapters.json;
import java.util.Arrays;
/**
* POJO definition used to decode JSON message.
*/
public class JsonJrdsTree {
/**
* indentifier
*/
public String identifier;
/**
* label
*/
public String label;
/**
* Items
*/
public JsonJrdsItem[] items;
@Override
public String toString() {
return "JsonJrdsTree{" +
"identifier='" + identifier + '\'' +
", label='" + label + '\'' +
", items=" + Arrays.toString(items) +
'}';
}
}
binjr/binjr-adapter-jrds/src/main/resources/META-INF/services/eu.binjr.core.data.adapters.DataAdapterInfo
+18
View File
@@ -0,0 +1,18 @@
#
# Copyright 2017-2018 Frederic Thevenet
#
# Licensed under the Apache License, Version 2.0 (the "License");
# you may not use this file except in compliance with the License.
# You may obtain a copy of the License at
#
# http://www.apache.org/licenses/LICENSE-2.0
#
# Unless required by applicable law or agreed to in writing, software
# distributed under the License is distributed on an "AS IS" BASIS,
# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
# See the License for the specific language governing permissions and
# limitations under the License.
#
# JRDS Data adapter service implementation
eu.binjr.sources.jrds.adapters.JrdsDataAdapterInfo
binjr/binjr-adapter-logs/README.md
+5
View File
@@ -0,0 +1,5 @@
# binjr-adapter-logs
[![Maven Central](https://img.shields.io/maven-central/v/eu.binjr/binjr-adapter-logs.svg?label=Maven%20Central&style=flat-square)](https://search.maven.org/search?q=g:%22eu.binjr%22%20AND%20a:%22binjr-adapter-logs%22)
This module implements a DataAdapter capable of consuming data from text-based log files.
binjr/binjr-adapter-logs/build.gradle
+32
View File
@@ -0,0 +1,32 @@
/*
* Copyright 2020-2022 Frederic Thevenet
*
* Licensed under the Apache License, Version 2.0 (the "License");
* you may not use this file except in compliance with the License.
* You may obtain a copy of the License at
*
* http://www.apache.org/licenses/LICENSE-2.0
*
* Unless required by applicable law or agreed to in writing, software
* distributed under the License is distributed on an "AS IS" BASIS,
* WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
* See the License for the specific language governing permissions and
* limitations under the License.
*/
dependencies {
compileOnly project(':binjr-core')
testCompileOnly project(':binjr-core')
}
jar {
manifest {
attributes(
'Specification-Title': project.name,
'Specification-Version': project.version,
'Implementation-Title': project.name,
'Implementation-Version': project.version,
'Build-Number': BINJR_BUILD_NUMBER
)
}
}
binjr/binjr-adapter-logs/src/main/java/eu/binjr/sources/logs/adapters/LogsAdapterPreferences.java
+80
View File
@@ -0,0 +1,80 @@
/*
* Copyright 2020 Frederic Thevenet
*
* Licensed under the Apache License, Version 2.0 (the "License");
* you may not use this file except in compliance with the License.
* You may obtain a copy of the License at
*
* http://www.apache.org/licenses/LICENSE-2.0
*
* Unless required by applicable law or agreed to in writing, software
* distributed under the License is distributed on an "AS IS" BASIS,
* WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
* See the License for the specific language governing permissions and
* limitations under the License.
*/
package eu.binjr.sources.logs.adapters;
import com.google.gson.Gson;
import eu.binjr.common.preferences.ObservablePreference;
import eu.binjr.core.data.adapters.DataAdapter;
import eu.binjr.core.data.adapters.DataAdapterPreferences;
import eu.binjr.core.data.indexes.parser.profile.BuiltInParsingProfile;
import eu.binjr.core.data.indexes.parser.profile.ParsingProfile;
import java.nio.charset.StandardCharsets;
/**
* Defines the preferences associated with the Log files adapter.
*
* @author Frederic Thevenet
*/
public class LogsAdapterPreferences extends DataAdapterPreferences {
private static final Gson gson = new Gson();
/**
* The default text panel font size preference.
*/
public ObservablePreference<Number> defaultTextViewFontSize = integerPreference("defaultTextViewFontSize", 10);
/**
* The filters used when scanning folders in the source filesystem.
*/
public ObservablePreference<String[]> folderFilters = objectPreference(String[].class,
"folderFilters",
new String[]{"*"},
gson::toJson,
s -> gson.fromJson(s, String[].class));
/**
* The filters used to prune file extensions to scan in the source filesystem.
*/
public ObservablePreference<String[]> fileExtensionFilters = objectPreference(String[].class,
"extensionFilters",
new String[]{"*.log", "*.txt"},
gson::toJson,
s -> gson.fromJson(s, String[].class));
/**
* The most recently used {@link ParsingProfile}
*/
public ObservablePreference<String> mostRecentlyUsedParsingProfile =
stringPreference("mostRecentlyUsedParsingProfile", BuiltInParsingProfile.ISO.getProfileId());
/**
* The most recently used encoding
*/
public ObservablePreference<String> mruEncoding =
stringPreference("mruEncoding", StandardCharsets.UTF_8.name());
/**
* Initialize a new instance of the {@link LogsAdapterPreferences} class associated to
* a {@link DataAdapter} instance.
*
* @param dataAdapterClass the associated {@link DataAdapter}
*/
public LogsAdapterPreferences(Class<? extends DataAdapter<?>> dataAdapterClass) {
super(dataAdapterClass);
}
}
binjr/binjr-adapter-logs/src/main/java/eu/binjr/sources/logs/adapters/LogsDataAdapter.java
+473
View File
@@ -0,0 +1,473 @@
/*
* Copyright 2020-2023 Frederic Thevenet
*
* Licensed under the Apache License, Version 2.0 (the "License");
* you may not use this file except in compliance with the License.
* You may obtain a copy of the License at
*
* http://www.apache.org/licenses/LICENSE-2.0
*
* Unless required by applicable law or agreed to in writing, software
* distributed under the License is distributed on an "AS IS" BASIS,
* WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
* See the License for the specific language governing permissions and
* limitations under the License.
*/
package eu.binjr.sources.logs.adapters;
import com.google.gson.Gson;
import eu.binjr.common.function.CheckedFunction;
import eu.binjr.common.function.CheckedLambdas;
import eu.binjr.common.io.FileSystemBrowser;
import eu.binjr.common.io.IOUtils;
import eu.binjr.common.javafx.controls.TimeRange;
import eu.binjr.common.javafx.controls.TreeViewUtils;
import eu.binjr.common.logging.Logger;
import eu.binjr.common.logging.Profiler;
import eu.binjr.common.preferences.MostRecentlyUsedList;
import eu.binjr.common.text.BinaryPrefixFormatter;
import eu.binjr.core.data.adapters.*;
import eu.binjr.core.data.exceptions.CannotInitializeDataAdapterException;
import eu.binjr.core.data.exceptions.DataAdapterException;
import eu.binjr.core.data.indexes.*;
import eu.binjr.core.data.indexes.parser.EventFormat;
import eu.binjr.core.data.indexes.parser.LogEventFormat;
import eu.binjr.core.data.indexes.parser.profile.CustomParsingProfile;
import eu.binjr.core.data.indexes.parser.profile.ParsingProfile;
import eu.binjr.core.data.timeseries.TimeSeriesProcessor;
import eu.binjr.core.data.workspace.LogFileSeriesInfo;
import eu.binjr.core.data.workspace.TimeSeriesInfo;
import eu.binjr.core.dialogs.Dialogs;
import eu.binjr.core.preferences.UserHistory;
import javafx.beans.property.*;
import javafx.beans.value.ChangeListener;
import org.apache.lucene.document.Field;
import org.apache.lucene.document.StoredField;
import org.apache.lucene.document.TextField;
import org.apache.lucene.facet.FacetField;
import org.eclipse.fx.ui.controls.tree.FilterableTreeItem;
import java.io.BufferedReader;
import java.io.IOException;
import java.io.InputStream;
import java.io.InputStreamReader;
import java.nio.charset.Charset;
import java.nio.charset.StandardCharsets;
import java.nio.file.Path;
import java.nio.file.Paths;
import java.time.Instant;
import java.time.ZoneId;
import java.util.*;
import java.util.stream.Collectors;
import static eu.binjr.core.data.indexes.parser.capture.CaptureGroup.SEVERITY;
/**
* A {@link DataAdapter} implementation to retrieve data from a text file.
*
* @author Frederic Thevenet
*/
public class LogsDataAdapter extends BaseDataAdapter<SearchHit> implements ProgressAdapter<SearchHit> {
private static final Logger logger = Logger.create(LogsDataAdapter.class);
private static final Gson gson = new Gson();
private static final String DEFAULT_PREFIX = "[Logs]";
private static final String ZONE_ID_PARAM_NAME = "zoneId";
private static final String ROOT_PATH_PARAM_NAME = "rootPath";
private static final String FOLDER_FILTERS_PARAM_NAME = "folderFilters";
private static final String EXTENSIONS_FILTERS_PARAM_NAME = "fileExtensionsFilters";
private static final String PARSING_PROFILE_PARAM_NAME = "parsingProfile";
private static final String LOG_FILE_ENCODING = "LOG_FILE_ENCODING";
private static final Property<IndexingStatus> INDEXING_OK = new ReadOnlyObjectWrapper(new SimpleObjectProperty<>(IndexingStatus.OK));
private final String sourceNamePrefix;
private final Map<String, IndexingStatus> indexedFiles = new HashMap<>();
private final BinaryPrefixFormatter binaryPrefixFormatter = new BinaryPrefixFormatter("###,###.## ");
private final MostRecentlyUsedList<String> defaultParsingProfiles =
UserHistory.getInstance().stringMostRecentlyUsedList("defaultParsingProfiles", 100);
private final MostRecentlyUsedList<String> userParsingProfiles =
UserHistory.getInstance().stringMostRecentlyUsedList("userParsingProfiles", 100);
private final Charset encoding;
private Path rootPath;
private Indexable index;
private FileSystemBrowser fileBrowser;
private String[] folderFilters;
private String[] fileExtensionsFilters;
private ParsingProfile parsingProfile;
private EventFormat<InputStream> defaultEventFormat;
private ZoneId zoneId;
/**
* Initializes a new instance of the {@link LogsDataAdapter} class.
*
* @throws DataAdapterException if an error occurs while initializing the adapter.
*/
public LogsDataAdapter() throws DataAdapterException {
super();
zoneId = ZoneId.systemDefault();
encoding = StandardCharsets.UTF_8;
sourceNamePrefix = DEFAULT_PREFIX;
}
/**
* Initializes a new instance of the {@link LogsDataAdapter} class from the provided {@link Path}
*
* @param rootPath the {@link Path} from which to load content.
* @param folderFilters a list of names of folders to inspect for content.
* @param fileExtensionsFilters a list of file extensions to inspect for content.
* @param profile the parsing profile to use.
* @throws DataAdapterException if an error occurs initializing the adapter.
*/
public LogsDataAdapter(Path rootPath,
ZoneId zoneId,
String[] folderFilters,
String[] fileExtensionsFilters,
ParsingProfile profile) throws DataAdapterException {
this(DEFAULT_PREFIX, rootPath, zoneId, StandardCharsets.UTF_8, folderFilters, fileExtensionsFilters, profile);
}
/**
* Initializes a new instance of the {@link LogsDataAdapter} class from the provided {@link Path}
*
* @param sourcePrefix the name to prepend the source with.
* @param rootPath the {@link Path} from which to load content.
* @param folderFilters a list of names of folders to inspect for content.
* @param fileExtensionsFilters a list of file extensions to inspect for content.
* @param profile the parsing profile to use.
* @throws DataAdapterException if an error occurs initializing the adapter.
*/
public LogsDataAdapter(String sourcePrefix,
Path rootPath,
ZoneId zoneId,
String[] folderFilters,
String[] fileExtensionsFilters,
ParsingProfile profile) throws DataAdapterException {
this(sourcePrefix, rootPath, zoneId, StandardCharsets.UTF_8, folderFilters, fileExtensionsFilters, profile);
}
/**
* Initializes a new instance of the {@link LogsDataAdapter} class from the provided {@link Path}
*
* @param rootPath the {@link Path} from which to load content.
* @param folderFilters a list of names of folders to inspect for content.
* @param fileExtensionsFilters a list of file extensions to inspect for content.
* @param profile the parsing profile to use.
* @throws DataAdapterException if an error occurs initializing the adapter.
*/
public LogsDataAdapter(Path rootPath,
ZoneId zoneId,
Charset encoding,
String[] folderFilters,
String[] fileExtensionsFilters,
ParsingProfile profile) throws DataAdapterException {
this(DEFAULT_PREFIX, rootPath, zoneId, encoding, folderFilters, fileExtensionsFilters, profile);
}
/**
* Initializes a new instance of the {@link LogsDataAdapter} class from the provided {@link Path}
*
* @param sourcePrefix the name to prepend the source with.
* @param rootPath the {@link Path} from which to load content.
* @param folderFilters a list of names of folders to inspect for content.
* @param fileExtensionsFilters a list of file extensions to inspect for content.
* @param profile the parsing profile to use.
* @throws DataAdapterException if an error occurs initializing the adapter.
*/
public LogsDataAdapter(String sourcePrefix,
Path rootPath,
ZoneId zoneId,
Charset encoding,
String[] folderFilters,
String[] fileExtensionsFilters,
ParsingProfile profile) throws DataAdapterException {
super();
this.sourceNamePrefix = sourcePrefix;
this.rootPath = rootPath;
this.encoding = encoding;
Map<String, String> params = new HashMap<>();
initParams(rootPath, zoneId, folderFilters, fileExtensionsFilters, profile);
}
@Override
public Map<String, String> getParams() {
Map<String, String> params = new HashMap<>();
params.put(LOG_FILE_ENCODING, getEncoding());
params.put(ROOT_PATH_PARAM_NAME, rootPath.toString());
params.put(ZONE_ID_PARAM_NAME, zoneId.toString());
params.put(FOLDER_FILTERS_PARAM_NAME, gson.toJson(folderFilters));
params.put(EXTENSIONS_FILTERS_PARAM_NAME, gson.toJson(fileExtensionsFilters));
params.put(PARSING_PROFILE_PARAM_NAME, gson.toJson(CustomParsingProfile.of(parsingProfile)));
return params;
}
@Override
public void loadParams(Map<String, String> params) throws DataAdapterException {
if (logger.isDebugEnabled()) {
logger.debug(() -> "LogsDataAdapter params:");
params.forEach((s, s2) -> logger.debug(() -> "key=" + s + ", value=" + s2));
}
initParams(Paths.get(validateParameterNullity(params, ROOT_PATH_PARAM_NAME)),
validateParameter(params, ZONE_ID_PARAM_NAME,
s -> {
if (s == null) {
logger.warn("Parameter " + ZONE_ID_PARAM_NAME + " is missing in adapter " + getSourceName());
return ZoneId.systemDefault();
}
return ZoneId.of(s);
}),
gson.fromJson(validateParameterNullity(params, FOLDER_FILTERS_PARAM_NAME), String[].class),
gson.fromJson(validateParameterNullity(params, EXTENSIONS_FILTERS_PARAM_NAME), String[].class),
gson.fromJson(validateParameterNullity(params, PARSING_PROFILE_PARAM_NAME), CustomParsingProfile.class));
}
private void initParams(Path rootPath,
ZoneId zoneId,
String[] folderFilters,
String[] fileExtensionsFilters,
ParsingProfile parsingProfile) throws DataAdapterException {
this.rootPath = rootPath;
this.zoneId = zoneId;
this.folderFilters = folderFilters;
this.fileExtensionsFilters = fileExtensionsFilters;
this.parsingProfile = parsingProfile;
this.defaultEventFormat = new LogEventFormat(parsingProfile, getTimeZoneId(), encoding);
}
@Override
public void onStart() throws DataAdapterException {
super.onStart();
try {
this.fileBrowser = FileSystemBrowser.of(rootPath);
this.index = Indexes.LOG_FILES.acquire();
} catch (IOException e) {
throw new CannotInitializeDataAdapterException("An error occurred during the data adapter initialization", e);
}
}
@Override
public FilterableTreeItem<SourceBinding> getBindingTree() throws DataAdapterException {
FilterableTreeItem<SourceBinding> configNode = new FilterableTreeItem<>(
new LogFilesBinding.Builder()
.withLabel(getSourceName())
.withAdapter(this)
.build());
attachNodes(configNode);
return configNode;
}
private void attachNodes(FilterableTreeItem<SourceBinding> root) throws DataAdapterException {
try (var p = Profiler.start("Building log files binding tree", logger::perf)) {
Map<Path, FilterableTreeItem<SourceBinding>> nodeDict = new HashMap<>();
nodeDict.put(fileBrowser.toInternalPath("/"), root);
for (var fsEntry : fileBrowser.listEntries(folderFilters, fileExtensionsFilters)) {
String fileName = fsEntry.getPath().getFileName().toString();
var attachTo = root;
if (fsEntry.getPath().getParent() != null) {
attachTo = nodeDict.get(fsEntry.getPath().getParent());
if (attachTo == null) {
attachTo = makeBranchNode(nodeDict, fsEntry.getPath().getParent(), root);
}
}
FilterableTreeItem<SourceBinding> filenode = new FilterableTreeItem<>(
new LogFilesBinding.Builder()
.withLabel(fileName + " (" + binaryPrefixFormatter.format(fsEntry.getSize()) + "B)")
.withPath(getId() + "/" + fsEntry.getPath().toString())
.withParent(attachTo.getValue())
.withParsingProfile(parsingProfile)
.withAdapter(this)
.build());
attachTo.getInternalChildren().add(filenode);
}
TreeViewUtils.sortFromBranch(root);
} catch (Exception e) {
Dialogs.notifyException("Error while enumerating files: " + e.getMessage(), e);
}
}
private FilterableTreeItem<SourceBinding> makeBranchNode(Map<Path, FilterableTreeItem<SourceBinding>> nodeDict,
Path path,
FilterableTreeItem<SourceBinding> root) {
var parent = root;
var rootPath = path.isAbsolute() ? path.getRoot() : path.getName(0);
for (int i = 0; i < path.getNameCount(); i++) {
Path current = rootPath.resolve(path.getName(i));
FilterableTreeItem<SourceBinding> filenode = nodeDict.get(current);
if (filenode == null) {
filenode = new FilterableTreeItem<>(
new LogFilesBinding.Builder()
.withLabel(current.getFileName().toString())
.withPath(getId() + "/" + path.toString())
.withParent(parent.getValue())
.withParsingProfile(parsingProfile)
.withAdapter(this)
.build());
nodeDict.put(current, filenode);
parent.getInternalChildren().add(filenode);
}
parent = filenode;
rootPath = current;
}
return parent;
}
@Override
public TimeRange getInitialTimeRange(String path, List<TimeSeriesInfo<SearchHit>> seriesInfo) throws DataAdapterException {
try {
return index.getTimeRangeBoundaries(seriesInfo.stream().map(this::getPathFacetValue).toList(), getTimeZoneId());
} catch (IOException e) {
throw new DataAdapterException("Error retrieving initial time range", e);
}
}
@Deprecated
@Override
public Map<TimeSeriesInfo<SearchHit>, TimeSeriesProcessor<SearchHit>> fetchData(String path,
Instant begin,
Instant end,
List<TimeSeriesInfo<SearchHit>> seriesInfo,
boolean bypassCache) throws DataAdapterException {
return loadSeries(path, seriesInfo, bypassCache ? ReloadPolicy.ALL : ReloadPolicy.UNLOADED, null, INDEXING_OK);
}
@Override
public Map<TimeSeriesInfo<SearchHit>, TimeSeriesProcessor<SearchHit>> loadSeries(String path,
List<TimeSeriesInfo<SearchHit>> seriesInfo,
ReloadPolicy reloadPolicy,
DoubleProperty progress,
Property<IndexingStatus> indexingStatus) throws DataAdapterException {
Map<TimeSeriesInfo<SearchHit>, TimeSeriesProcessor<SearchHit>> data = new HashMap<>();
try {
ensureIndexed(seriesInfo.stream()
.filter(s -> s instanceof LogFileSeriesInfo)
.map(s -> (LogFileSeriesInfo) s)
.toList(),
progress,
reloadPolicy,
indexingStatus);
} catch (Exception e) {
throw new DataAdapterException("Error fetching logs from " + path, e);
}
return data;
}
private synchronized void ensureIndexed(List<LogFileSeriesInfo> seriesInfo,
DoubleProperty progress,
ReloadPolicy reloadPolicy,
Property<IndexingStatus> indexingStatus) throws IOException {
final var toDo = seriesInfo.stream()
.filter(p -> switch (reloadPolicy) {
case ALL -> true;
case UNLOADED -> !indexedFiles.containsKey(getPathFacetValue(p));
case INCOMPLETE ->
indexedFiles.getOrDefault(getPathFacetValue(p), IndexingStatus.CANCELED) == IndexingStatus.CANCELED;
})
.toList();
if (toDo.size() > 0) {
final long totalSizeInBytes = toDo.stream()
.map(CheckedLambdas.wrap((CheckedFunction<LogFileSeriesInfo, Long, IOException>)
e -> fileBrowser.getEntry(e.getBinding().getPath().replace(getId() + "/", "")).getSize()))
.reduce(Long::sum).orElse(0L);
final ChangeListener<Number> progressListener = (observable, oldValue, newValue) -> {
if (newValue != null && totalSizeInBytes > 0) {
var oldProgress = (oldValue.longValue() * 100 / totalSizeInBytes) / 100.0;
var newProgress = (newValue.longValue() * 100 / totalSizeInBytes) / 100.0;
if (progress != null && oldProgress != newProgress) {
Dialogs.runOnFXThread(() -> progress.setValue(newProgress));
}
}
};
final LongProperty charRead = new SimpleLongProperty(0);
charRead.addListener(progressListener);
try {
for (int i = 0; i < toDo.size(); i++) {
var tsInfo = toDo.get(i);
String path = tsInfo.getBinding().getPath();
var key = getPathFacetValue(tsInfo);
index.add(key,
fileBrowser.getData(path.replace(getId() + "/", "")),
(i == toDo.size() - 1), // commit if last file
getEventFormat(tsInfo),
(doc, event) -> {
// add all other sections as prefixed search fields
event.getTextFields().entrySet().stream()
.filter(e -> !e.getKey().equals(SEVERITY))
.forEach(e -> doc.add(new TextField(e.getKey(), e.getValue(), Field.Store.NO)));
// Add severity
String severity = event.getTextField(SEVERITY) == null ? "unknown" : event.getTextField(SEVERITY).toLowerCase();
doc.add(new FacetField(SEVERITY, severity));
doc.add(new StoredField(SEVERITY, severity));
return doc;
},
charRead,
indexingStatus);
indexedFiles.put(key, indexingStatus.getValue());
}
} finally {
// remove listener
charRead.removeListener(progressListener);
if (progress != null) {
Dialogs.runOnFXThread(() -> progress.setValue(-1));
}
// reset cancellation request
indexingStatus.setValue(IndexingStatus.OK);
}
}
// Update loading status for series
for (var series : seriesInfo) {
series.setIndexingStatus(indexedFiles.get(getPathFacetValue(series)));
}
}
private String readTextFile(String path) throws IOException {
try (Profiler ignored = Profiler.start("Extracting text from file " + path, logger::perf)) {
try (var reader = new BufferedReader(new InputStreamReader(fileBrowser.getData(path), StandardCharsets.UTF_8))) {
return reader.lines().collect(Collectors.joining("\n"));
}
}
}
private String getPathFacetValue(TimeSeriesInfo<?> p) {
if (p instanceof LogFileSeriesInfo lfsi && lfsi.getParsingProfile() != null) {
return lfsi.getPathFacetValue();
}
return LogFileSeriesInfo.makePathFacetValue(parsingProfile, p);
}
private EventFormat<InputStream> getEventFormat(TimeSeriesInfo<?> p) {
if (p instanceof LogFileSeriesInfo lfsi && lfsi.getParsingProfile() != null) {
return new LogEventFormat(lfsi.getParsingProfile(), getTimeZoneId(), encoding);
}
return defaultEventFormat;
}
@Override
public String getEncoding() {
return encoding.name();
}
@Override
public ZoneId getTimeZoneId() {
return zoneId;
}
@Override
public String getSourceName() {
return String.format("%s %s", sourceNamePrefix, rootPath != null ? rootPath.getFileName() : "???");
}
@Override
public void close() {
try {
Indexes.LOG_FILES.release();
} catch (Exception e) {
logger.error("An error occurred while releasing index " + Indexes.LOG_FILES.name() + ": " + e.getMessage());
logger.debug("Stack Trace:", e);
}
IOUtils.close(fileBrowser);
super.close();
}
}
binjr/binjr-adapter-logs/src/main/java/eu/binjr/sources/logs/adapters/LogsDataAdapterDialog.java
+221
View File
@@ -0,0 +1,221 @@
/*
* Copyright 2020-2022 Frederic Thevenet
*
* Licensed under the Apache License, Version 2.0 (the "License");
* you may not use this file except in compliance with the License.
* You may obtain a copy of the License at
*
* http://www.apache.org/licenses/LICENSE-2.0
*
* Unless required by applicable law or agreed to in writing, software
* distributed under the License is distributed on an "AS IS" BASIS,
* WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
* See the License for the specific language governing permissions and
* limitations under the License.
*/
package eu.binjr.sources.logs.adapters;
import eu.binjr.common.javafx.controls.LabelWithInlineHelp;
import eu.binjr.common.javafx.controls.NodeUtils;
import eu.binjr.common.logging.Logger;
import eu.binjr.core.data.adapters.DataAdapter;
import eu.binjr.core.data.adapters.DataAdapterFactory;
import eu.binjr.core.data.exceptions.CannotInitializeDataAdapterException;
import eu.binjr.core.data.exceptions.DataAdapterException;
import eu.binjr.core.data.exceptions.NoAdapterFoundException;
import eu.binjr.core.data.indexes.parser.profile.BuiltInParsingProfile;
import eu.binjr.core.data.indexes.parser.profile.ParsingProfile;
import eu.binjr.core.dialogs.DataAdapterDialog;
import eu.binjr.core.dialogs.Dialogs;
import eu.binjr.core.dialogs.LogParsingProfileDialog;
import eu.binjr.core.preferences.UserPreferences;
import javafx.geometry.*;
import javafx.scene.Node;
import javafx.scene.control.*;
import javafx.scene.layout.GridPane;
import javafx.scene.layout.HBox;
import javafx.scene.layout.Priority;
import javafx.stage.DirectoryChooser;
import javafx.stage.FileChooser;
import org.controlsfx.control.textfield.TextFields;
import java.io.File;
import java.nio.charset.Charset;
import java.nio.file.Files;
import java.nio.file.Path;
import java.nio.file.Paths;
import java.time.ZoneId;
import java.util.Arrays;
import java.util.Collection;
import java.util.List;
import java.util.Objects;
/**
* An implementation of the {@link DataAdapterDialog} class that presents a dialog box to retrieve the parameters specific {@link LogsDataAdapterDialog}
*
* @author Frederic Thevenet
*/
public class LogsDataAdapterDialog extends DataAdapterDialog<Path> {
private static final Logger logger = Logger.create(LogsDataAdapterDialog.class);
private final TextField extensionFiltersTextField;
private final TextField encodingField;
private final LogsAdapterPreferences prefs;
private final ChoiceBox<ParsingProfile> parsingChoiceBox = new ChoiceBox<>();
private void updateProfileList(ParsingProfile[] newValue) {
parsingChoiceBox.getItems().clear();
parsingChoiceBox.getItems().setAll(BuiltInParsingProfile.editableProfiles());
parsingChoiceBox.getItems().addAll(newValue);
parsingChoiceBox.getSelectionModel().select(parsingChoiceBox.getItems().stream()
.filter(p -> Objects.equals(p.getProfileId(), prefs.mostRecentlyUsedParsingProfile.get()))
.findAny().orElse(BuiltInParsingProfile.ALL));
}
/**
* Initializes a new instance of the {@link LogsDataAdapterDialog} class.
*
* @param owner the owner window for the dialog
* @throws NoAdapterFoundException if an error occurs initializing the dialog.
*/
public LogsDataAdapterDialog(Node owner) throws NoAdapterFoundException {
super(owner, Mode.PATH, "mostRecentLogsArchives", true);
this.prefs = (LogsAdapterPreferences) DataAdapterFactory.getInstance().getAdapterPreferences(LogsDataAdapter.class.getName());
setDialogHeaderText("Add a Log File, Zip Archive or Folder");
this.setUriLabelInlineHelp("""
A file system path to get log files from.
It can either be:
- The path to a single text-based log file, which will then be the sole item available in the source tree.
- The path to a folder, in which case all files in the folder and sub-folders that match the extension filter below will be available the the source tree.
- The path to a Zip archive, in which case all files in the archive that match the extension filter below will be available the the source tree.
""");
this.setTimezoneLabelInlineHelp("""
The default timezone for timestamps in the source files.
""");
extensionFiltersTextField = new TextField(String.join(", ", prefs.fileExtensionFilters.get()));
var extensionlabel = new LabelWithInlineHelp();
extensionlabel.setText("Extensions");
extensionlabel.setInlineHelp("""
The set of extensions used to filter which files should be available in the source tree when pointing to a folder or a zip archive.
You can specify any number of glob patterns separated by spaces, commas or semi-colons.
""");
extensionlabel.setAlignment(Pos.CENTER_RIGHT);
GridPane.setConstraints(extensionlabel, 0, 2, 1, 1, HPos.LEFT, VPos.CENTER, Priority.ALWAYS, Priority.ALWAYS, new Insets(4, 0, 4, 0));
GridPane.setConstraints(extensionFiltersTextField, 1, 2, 1, 1, HPos.LEFT, VPos.CENTER, Priority.ALWAYS, Priority.ALWAYS, new Insets(4, 0, 4, 0));
this.encodingField = new TextField(prefs.mruEncoding.get());
TextFields.bindAutoCompletion(encodingField, Charset.availableCharsets().keySet());
GridPane.setConstraints(encodingField, 1, 3, 1, 1, HPos.LEFT, VPos.CENTER, Priority.ALWAYS, Priority.ALWAYS, new Insets(4, 0, 4, 0));
var encodingLabel = new LabelWithInlineHelp();
encodingLabel.setText("Encoding");
encodingLabel.setInlineHelp("""
The text encoding of files to extracts log events from.
""");
encodingLabel.setAlignment(Pos.CENTER_RIGHT);
GridPane.setConstraints(encodingLabel, 0, 3, 1, 1, HPos.LEFT, VPos.CENTER, Priority.ALWAYS, Priority.ALWAYS, new Insets(4, 0, 4, 0));
var parsingLabel = new LabelWithInlineHelp();
parsingLabel.setText("Parsing profile");
parsingLabel.setInlineHelp("""
The default parsing profile used to process log files loaded from this source.
It is still possible to select a different profile on of file-by-file basis once the source is loaded.
""");
parsingLabel.setAlignment(Pos.CENTER_RIGHT);
var parsingHBox = new HBox();
parsingHBox.setSpacing(5);
updateProfileList(UserPreferences.getInstance().userLogEventsParsingProfiles.get());
parsingChoiceBox.setMaxWidth(Double.MAX_VALUE);
var editParsingButton = new Button("Edit");
editParsingButton.setOnAction(event -> {
try {
new LogParsingProfileDialog(this.getOwner(), parsingChoiceBox.getValue()).showAndWait().ifPresent(selection -> {
prefs.mostRecentlyUsedParsingProfile.set(selection.getProfileId());
updateProfileList( UserPreferences.getInstance().userLogEventsParsingProfiles.get());
});
} catch (Exception e) {
Dialogs.notifyException("Failed to show parsing profile windows", e, owner);
}
});
parsingHBox.getChildren().addAll(parsingChoiceBox, editParsingButton);
HBox.setHgrow(parsingChoiceBox, Priority.ALWAYS);
GridPane.setConstraints(parsingLabel, 0, 4, 1, 1, HPos.LEFT, VPos.CENTER, Priority.ALWAYS, Priority.ALWAYS, new Insets(4, 0, 4, 0));
GridPane.setConstraints(parsingHBox, 1, 4, 1, 1, HPos.LEFT, VPos.CENTER, Priority.ALWAYS, Priority.ALWAYS, new Insets(4, 0, 4, 0));
getParamsGridPane().getChildren().addAll(extensionlabel, extensionFiltersTextField, parsingLabel, parsingHBox, encodingLabel, encodingField);
}
@Override
protected File displayFileChooser(Node owner) {
try {
ContextMenu sourceMenu = new ContextMenu();
MenuItem fileMenuItem = new MenuItem("Log file");
fileMenuItem.setOnAction(eventHandler -> {
FileChooser fileChooser = new FileChooser();
fileChooser.setTitle("Open Log File");
fileChooser.getExtensionFilters().add(new FileChooser.ExtensionFilter("Log file", "*.log", "*.txt", "*.trace", "*.out", "*.err"));
fileChooser.getExtensionFilters().add(new FileChooser.ExtensionFilter("All files", "*.*", "*"));
Dialogs.getInitialDir(getMostRecentList()).ifPresent(fileChooser::setInitialDirectory);
File selectedFile = fileChooser.showOpenDialog(NodeUtils.getStage(owner));
if (selectedFile != null) {
setSourceUri(selectedFile.getPath());
}
});
sourceMenu.getItems().add(fileMenuItem);
MenuItem zipMenuItem = new MenuItem("Zip file");
zipMenuItem.setOnAction(eventHandler -> {
FileChooser fileChooser = new FileChooser();
fileChooser.setTitle("Open Zip Archive");
fileChooser.getExtensionFilters().add(new FileChooser.ExtensionFilter("Zip archive", "*.zip"));
fileChooser.getExtensionFilters().add(new FileChooser.ExtensionFilter("All files", "*.*", "*"));
Dialogs.getInitialDir(getMostRecentList()).ifPresent(fileChooser::setInitialDirectory);
File selectedFile = fileChooser.showOpenDialog(NodeUtils.getStage(owner));
if (selectedFile != null) {
setSourceUri(selectedFile.getPath());
}
});
sourceMenu.getItems().add(zipMenuItem);
MenuItem folderMenuItem = new MenuItem("Folder");
folderMenuItem.setOnAction(eventHandler -> {
DirectoryChooser dirChooser = new DirectoryChooser();
dirChooser.setTitle("Open Folder");
Dialogs.getInitialDir(getMostRecentList()).ifPresent(dirChooser::setInitialDirectory);
File selectedFile = dirChooser.showDialog(NodeUtils.getStage(owner));
if (selectedFile != null) {
setSourceUri(selectedFile.getPath());
}
});
sourceMenu.getItems().add(folderMenuItem);
sourceMenu.show(owner, Side.RIGHT, 0, 0);
} catch (Exception e) {
Dialogs.notifyException("Error while displaying file chooser: " + e.getMessage(), e, owner);
}
return null;
}
@Override
protected Collection<DataAdapter> getDataAdapters() throws DataAdapterException {
Path path = Paths.get(getSourceUri());
if (!Files.exists(path)) {
throw new CannotInitializeDataAdapterException("Cannot find " + getSourceUri());
}
if (!path.isAbsolute()) {
throw new CannotInitializeDataAdapterException("The provided path is not valid.");
}
getMostRecentList().push(path);
prefs.fileExtensionFilters.set(Arrays.stream(extensionFiltersTextField.getText().split("[,;\" ]+")).filter(e -> !e.isBlank()).toArray(String[]::new));
prefs.mostRecentlyUsedParsingProfile.set(parsingChoiceBox.getValue().getProfileId());
String charsetName = encodingField.getText();
if (!Charset.isSupported(charsetName)) {
throw new CannotInitializeDataAdapterException("Invalid or unsupported encoding: " + charsetName);
}
return List.of(new LogsDataAdapter(path,
ZoneId.of(getSourceTimezone()),
Charset.forName(charsetName),
prefs.folderFilters.get(),
prefs.fileExtensionFilters.get(),
parsingChoiceBox.getValue()));
}
}
binjr/binjr-adapter-logs/src/main/java/eu/binjr/sources/logs/adapters/LogsDataAdapterInfo.java
+57
View File
@@ -0,0 +1,57 @@
/*
* Copyright 2020-2023 Frederic Thevenet
*
* Licensed under the Apache License, Version 2.0 (the "License");
* you may not use this file except in compliance with the License.
* You may obtain a copy of the License at
*
* http://www.apache.org/licenses/LICENSE-2.0
*
* Unless required by applicable law or agreed to in writing, software
* distributed under the License is distributed on an "AS IS" BASIS,
* WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
* See the License for the specific language governing permissions and
* limitations under the License.
*/
package eu.binjr.sources.logs.adapters;
import eu.binjr.common.javafx.controls.ToolButtonBuilder;
import eu.binjr.core.data.adapters.AdapterMetadata;
import eu.binjr.core.data.adapters.BaseDataAdapterInfo;
import eu.binjr.core.data.adapters.SourceLocality;
import eu.binjr.core.data.adapters.VisualizationType;
import eu.binjr.core.data.exceptions.CannotInitializeDataAdapterException;
import eu.binjr.core.preferences.AppEnvironment;
/**
* Defines the metadata associated with LogsDataAdapterInfo.
*
* @author Frederic Thevenet
*/
@AdapterMetadata(
name = "Logs",
description = "Log Files Data Adapter",
copyright = AppEnvironment.COPYRIGHT_NOTICE,
license = AppEnvironment.LICENSE,
siteUrl = AppEnvironment.HTTP_WWW_BINJR_EU,
adapterClass = LogsDataAdapter.class,
dialogClass = LogsDataAdapterDialog.class,
preferencesClass = LogsAdapterPreferences.class,
sourceLocality = SourceLocality.LOCAL,
apiLevel = AppEnvironment.PLUGIN_API_LEVEL,
visualizationType = VisualizationType.EVENTS
)
public class LogsDataAdapterInfo extends BaseDataAdapterInfo {
/**
* Initialises a new instance of the {@link LogsDataAdapterInfo} class.
*
* @throws CannotInitializeDataAdapterException if the adapter's initialization failed
*/
public LogsDataAdapterInfo() throws CannotInitializeDataAdapterException {
super(LogsDataAdapterInfo.class);
}
}
binjr/binjr-adapter-logs/src/main/resources/META-INF/services/eu.binjr.core.data.adapters.DataAdapterInfo
+18
View File
@@ -0,0 +1,18 @@
#
# Copyright 2020 Frederic Thevenet
#
# Licensed under the Apache License, Version 2.0 (the "License");
# you may not use this file except in compliance with the License.
# You may obtain a copy of the License at
#
# http://www.apache.org/licenses/LICENSE-2.0
#
# Unless required by applicable law or agreed to in writing, software
# distributed under the License is distributed on an "AS IS" BASIS,
# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
# See the License for the specific language governing permissions and
# limitations under the License.
#
# Text Data adapter service implementation
eu.binjr.sources.logs.adapters.LogsDataAdapterInfo
binjr/binjr-adapter-logs/src/test/java/eu/binjr/sources/logs/data/parsers/ParserTest.java
+134
View File
@@ -0,0 +1,134 @@
/*
* Copyright 2022 Frederic Thevenet
*
* Licensed under the Apache License, Version 2.0 (the "License");
* you may not use this file except in compliance with the License.
* You may obtain a copy of the License at
*
* http://www.apache.org/licenses/LICENSE-2.0
*
* Unless required by applicable law or agreed to in writing, software
* distributed under the License is distributed on an "AS IS" BASIS,
* WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
* See the License for the specific language governing permissions and
* limitations under the License.
*/
package eu.binjr.sources.logs.data.parsers;
import eu.binjr.core.data.indexes.parser.LogEventFormat;
import eu.binjr.core.data.indexes.parser.profile.BuiltInParsingProfile;
import eu.binjr.core.data.indexes.parser.profile.ParsingProfile;
import org.junit.jupiter.api.Test;
import java.io.ByteArrayInputStream;
import java.nio.charset.StandardCharsets;
import java.time.ZoneId;
import static org.junit.jupiter.api.Assertions.assertFalse;
import static org.junit.jupiter.api.Assertions.assertTrue;
public class ParserTest {
@Test
public void parseBinjrLogsFull() {
assertTrue(parsingTest(BuiltInParsingProfile.ISO, "[2020/07/21-02:56:42.460] [info] Hello World!"));
assertTrue(parsingTest(BuiltInParsingProfile.ISO, "[2020-11-13 19:59:22.627] [INFO ] Hello world!"));
}
@Test
public void parseBinjrLogsNoFraction() {
assertTrue(parsingTest(BuiltInParsingProfile.ISO, "[2020/07/21-02:56:42] [info] Hello World!"));
assertTrue(parsingTest(BuiltInParsingProfile.ISO, "[2020-11-13 19:59:22] [INFO ] Hello world!"));
}
@Test
public void parseBinjrLogsUnknownSeverity() {
assertTrue(parsingTest(BuiltInParsingProfile.ISO, "[2020-11-13 19:59:22.627] [FOO ] Hello world!"));
}
@Test
public void parseBinjrLogsNoSeverity() {
assertTrue(parsingTest(BuiltInParsingProfile.ISO, "[2020-11-13 19:59:22.627] Hello world!"));
}
@Test
public void parseBinjrStrictFull() {
assertFalse(parsingTest(BuiltInParsingProfile.BINJR_STRICT, "[2020/07/21-02:56:42.460] [info] Hello World!"));
assertTrue(parsingTest(BuiltInParsingProfile.BINJR_STRICT, "[2020-11-13 19:59:22.627] [INFO ] Hello world!"));
}
@Test
public void parseBinjrStrictNoFraction() {
assertFalse(parsingTest(BuiltInParsingProfile.BINJR_STRICT, "[2020/07/21-02:56:42] [info] Hello World!"));
assertFalse(parsingTest(BuiltInParsingProfile.BINJR_STRICT, "[2020-11-13 19:59:22] [INFO ] Hello world!"));
}
@Test
public void parseBinjrStrictUnknownSeverity() {
assertFalse(parsingTest(BuiltInParsingProfile.BINJR_STRICT, "[2020-11-13 19:59:22.627] [FOO ] Hello world!"));
}
@Test
public void parseBinjrStrictNoSeverity() {
assertFalse(parsingTest(BuiltInParsingProfile.BINJR_STRICT, "[2020-11-13 19:59:22.627] Hello world!"));
}
@Test
public void parseGc() {
// Logs with uptime
assertTrue(parsingTest(BuiltInParsingProfile.JVM, "[54.144s][info][gc,phases ] GC(59) Phase 4: Compact heap 11.810ms"));
assertTrue(parsingTest(BuiltInParsingProfile.JVM, "[54.151s][info][gc,heap ] GC(59) Eden regions: 9->0(130)"));
assertTrue(parsingTest(BuiltInParsingProfile.JVM, "[54.151s][info][gc,heap ] GC(59) Survivor regions: 0->0(12)"));
assertTrue(parsingTest(BuiltInParsingProfile.JVM, "[54.151s][info][gc,heap ] GC(59) Old regions: 139->137"));
assertTrue(parsingTest(BuiltInParsingProfile.JVM, "[54.151s][info][gc,heap ] GC(59) Archive regions: 0->0"));
assertTrue(parsingTest(BuiltInParsingProfile.JVM, "[54.151s][info][gc,heap ] GC(59) Humongous regions: 10->10"));
assertTrue(parsingTest(BuiltInParsingProfile.JVM, "[54.151s][info][gc,metaspace ] GC(59) Metaspace: 48468K(49200K)->48468K(49200K) NonClass: 41947K(42416K)->41947K(42416K) Class: 6521K(6784K)->6521K(6784K)"));
assertTrue(parsingTest(BuiltInParsingProfile.JVM, "[54.151s][info][gc ] GC(59) Pause Full (System.gc()) 153M->143M(477M) 94.613ms"));
assertTrue(parsingTest(BuiltInParsingProfile.JVM, "[54.152s][info][gc,cpu ] GC(59) User=0.66s Sys=0.00s Real=0.10s"));
assertTrue(parsingTest(BuiltInParsingProfile.JVM, "[65.250s][info][gc,heap,exit ] Heap"));
assertTrue(parsingTest(BuiltInParsingProfile.JVM, "[65.250s][info][gc,heap,exit ] garbage-first heap total 488448K, used 163531K [0x0000000080000000, 0x0000000100000000)"));
assertTrue(parsingTest(BuiltInParsingProfile.JVM, "[65.250s][info][gc,heap,exit ] region size 1024K, 17 young (17408K), 0 survivors (0K)"));
assertTrue(parsingTest(BuiltInParsingProfile.JVM, "[65.250s][info][gc,heap,exit ] Metaspace used 48659K, capacity 49344K, committed 49456K, reserved 1091584K"));
assertTrue(parsingTest(BuiltInParsingProfile.JVM, "[65.250s][info][gc,heap,exit ] class space used 6554K, capacity 6778K, committed 6784K, reserved 1048576K"));
//Logs with time (ISO)
assertTrue(parsingTest(BuiltInParsingProfile.JVM, "[2023-02-22T06:46:55.093+0200][info ][safepoint ] Application time: 0.4142997 seconds"));
assertTrue(parsingTest(BuiltInParsingProfile.JVM, "[2023-02-22T06:46:55.094+0200][info ][safepoint ] Entering safepoint region: G1CollectForAllocation"));
assertTrue(parsingTest(BuiltInParsingProfile.JVM, "[2023-02-22T06:46:55.097+0200][info ][gc,start ] GC(846) Pause Young (Normal) (GCLocker Initiated GC)"));
assertTrue(parsingTest(BuiltInParsingProfile.JVM, "[2023-02-22T06:46:55.097+0200][info ][gc,task ] GC(846) Using 143 workers of 143 for evacuation"));
assertTrue(parsingTest(BuiltInParsingProfile.JVM, "[2023-02-22T06:46:55.190+0200][info ][gc,phases ] GC(846) Pre Evacuate Collection Set: 32.2ms"));
assertTrue(parsingTest(BuiltInParsingProfile.JVM, "[2023-02-22T06:46:55.190+0200][info ][gc,phases ] GC(846) Evacuate Collection Set: 43.3ms"));
assertTrue(parsingTest(BuiltInParsingProfile.JVM, "[2023-02-22T06:46:55.190+0200][info ][gc,phases ] GC(846) Post Evacuate Collection Set: 9.2ms"));
assertTrue(parsingTest(BuiltInParsingProfile.JVM, "[2023-02-22T06:46:55.190+0200][info ][gc,phases ] GC(846) Other: 8.9ms"));
assertTrue(parsingTest(BuiltInParsingProfile.JVM, "[2023-02-22T06:46:55.190+0200][info ][gc,heap ] GC(846) Eden regions: 14->0(949)"));
assertTrue(parsingTest(BuiltInParsingProfile.JVM, "[2023-02-22T06:46:55.190+0200][info ][gc,heap ] GC(846) Survivor regions: 10->11(960)"));
assertTrue(parsingTest(BuiltInParsingProfile.JVM, "[2023-02-22T06:46:55.190+0200][info ][gc,heap ] GC(846) Old regions: 7420->7420"));
assertTrue(parsingTest(BuiltInParsingProfile.JVM, "[2023-02-22T06:46:55.190+0200][info ][gc,heap ] GC(846) Humongous regions: 11634->11632"));
assertTrue(parsingTest(BuiltInParsingProfile.JVM, "[2023-02-22T06:46:55.190+0200][info ][gc,metaspace ] GC(846) Metaspace: 231059K(255744K)->231059K(255744K)"));
assertTrue(parsingTest(BuiltInParsingProfile.JVM, "[2023-02-22T06:46:55.190+0200][info ][gc ] GC(846) Pause Young (Normal) (GCLocker Initiated GC) 610464M->609992M(614400M) 92.765ms"));
assertTrue(parsingTest(BuiltInParsingProfile.JVM, "[2023-02-22T06:46:55.191+0200][info ][gc,cpu ] GC(846) User=4.18s Sys=0.03s Real=0.09s"));
assertTrue(parsingTest(BuiltInParsingProfile.JVM, "[2023-02-22T06:46:55.192+0200][info ][safepoint ] Leaving safepoint region"));
assertTrue(parsingTest(BuiltInParsingProfile.JVM, "[2023-02-22T06:46:55.192+0200][info ][safepoint ] Total time for which application threads were stopped: 0.0987139 seconds, Stopping threads took: 0.0008103 seconds"));
assertTrue(parsingTest(BuiltInParsingProfile.JVM, "[2023-02-22T06:46:55.224+0200][info ][safepoint ] Application time: 0.0321887 seconds"));
assertTrue(parsingTest(BuiltInParsingProfile.JVM, "[2023-02-22T06:46:55.286+0200][info ][safepoint ] Entering safepoint region: G1CollectForAllocation"));
assertTrue(parsingTest(BuiltInParsingProfile.JVM, "[2023-02-22T06:46:55.292+0200][debug][gc,jni ] Setting _needs_gc. Thread `\"VM Thread\" 62 locked."));
assertTrue(parsingTest(BuiltInParsingProfile.JVM, "[2023-02-22T06:46:55.292+0200][info ][safepoint ] Leaving safepoint region"));
assertTrue(parsingTest(BuiltInParsingProfile.JVM, "[2023-02-22T06:46:55.292+0200][info ][safepoint ] Total time for which application threads were stopped: 0.0685306 seconds, Stopping threads took: 0.0622806 seconds"));
assertTrue(parsingTest(BuiltInParsingProfile.JVM, "[2023-02-22T06:46:55.299+0200][debug][gc,jni ] Allocation failed. Thread stalled by JNI critical section. Thread \"ForkJoinPool-280-worker-45_CommitThrottle#commit: top level tasks for transactions\" 48 locked."));
assertTrue(parsingTest(BuiltInParsingProfile.JVM, "[2023-02-22T06:46:55.299+0200][debug][gc,jni ] Allocation failed. Thread stalled by JNI critical section. Thread \"ForkJoinPool-280-worker-111_CommitThrottle#commit: top level tasks for transactions\" 46 locked."));
assertTrue(parsingTest(BuiltInParsingProfile.JVM, "[2023-02-22T06:46:55.305+0200][debug][gc,jni ] Allocation failed. Thread stalled by JNI critical section. Thread \"ForkJoinPool-280-worker-25_CommitThrottle#commit: top level tasks for transactions\" 46 locked."));
}
private boolean parsingTest(ParsingProfile profile, String text) {
var p = new LogEventFormat(profile, ZoneId.systemDefault(), StandardCharsets.UTF_8);
var res = p.parse(new ByteArrayInputStream(text.getBytes(StandardCharsets.UTF_8)));
System.out.println(res);
return res.iterator().next() != null;
}
}
binjr/binjr-adapter-netdata/README.md
+10
View File
@@ -0,0 +1,10 @@
# binjr-adapter-netdata
[![Maven Central](https://img.shields.io/maven-central/v/eu.binjr/binjr-adapter-netdata.svg?label=Maven%20Central&style=flat-square)](https://search.maven.org/search?q=g:%22eu.binjr%22%20AND%20a:%22binjr-adapter-netdata%22)
This module implements a DataAdapter capable of consuming data from a [Netdata](https://www.netdata.cloud/) instance.
Using this DataAdapter, you can connect to a Netdata server via HTTP and use the flexibility offered by binjr to
navigate through the data.
binjr/binjr-adapter-netdata/build.gradle
+51
View File
@@ -0,0 +1,51 @@
/*
* Copyright 2020-2022 Frederic Thevenet
*
* Licensed under the Apache License, Version 2.0 (the "License");
* you may not use this file except in compliance with the License.
* You may obtain a copy of the License at
*
* http://www.apache.org/licenses/LICENSE-2.0
*
* Unless required by applicable law or agreed to in writing, software
* distributed under the License is distributed on an "AS IS" BASIS,
* WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
* See the License for the specific language governing permissions and
* limitations under the License.
*/
plugins {
id "org.openapi.generator" version "6.5.0"
}
dependencies {
compileOnly project(':binjr-core')
}
openApiValidate {
inputSpec = "${projectDir}/specs/netdata-openapi.json"
}
openApiGenerate {
generatorName = "java"
inputSpec = "${projectDir}/specs/netdata-openapi.json"
outputDir = "${buildDir}/generated"
apiPackage = "org.openapi.netdata.api"
invokerPackage = "org.openapi.netdata.invoker"
modelPackage = "org.openapi.netdata.model"
configOptions = [
dateLibrary: "java8"
]
}
jar {
manifest {
attributes(
'Specification-Title': project.name,
'Specification-Version': project.version,
'Implementation-Title': project.name,
'Implementation-Version': project.version,
'Build-Number': BINJR_BUILD_NUMBER
)
}
}
binjr/binjr-adapter-netdata/specs/netdata-openapi.json
+1841
View File
File diff suppressed because it is too large Load Diff
binjr/binjr-adapter-netdata/src/main/java/eu/binjr/sources/netdata/adapters/NetdataAdapter.java
+203
View File
@@ -0,0 +1,203 @@
/*
* Copyright 2020-2023 Frederic Thevenet
*
* Licensed under the Apache License, Version 2.0 (the "License");
* you may not use this file except in compliance with the License.
* You may obtain a copy of the License at
*
* http://www.apache.org/licenses/LICENSE-2.0
*
* Unless required by applicable law or agreed to in writing, software
* distributed under the License is distributed on an "AS IS" BASIS,
* WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
* See the License for the specific language governing permissions and
* limitations under the License.
*/
package eu.binjr.sources.netdata.adapters;
import com.google.gson.Gson;
import eu.binjr.common.javafx.controls.TimeRange;
import eu.binjr.common.logging.Logger;
import eu.binjr.core.data.adapters.*;
import eu.binjr.core.data.codec.Decoder;
import eu.binjr.core.data.codec.csv.CsvDecoder;
import eu.binjr.core.data.exceptions.CannotInitializeDataAdapterException;
import eu.binjr.core.data.exceptions.DataAdapterException;
import eu.binjr.core.data.timeseries.DoubleTimeSeriesProcessor;
import eu.binjr.core.data.workspace.ChartType;
import eu.binjr.core.data.workspace.TimeSeriesInfo;
import eu.binjr.core.preferences.UserPreferences;
import eu.binjr.sources.netdata.api.Chart;
import eu.binjr.sources.netdata.api.ChartSummary;
import org.eclipse.fx.ui.controls.tree.FilterableTreeItem;
import java.net.URI;
import java.net.URL;
import java.nio.charset.StandardCharsets;
import java.time.Instant;
import java.time.ZoneId;
import java.time.ZonedDateTime;
import java.util.ArrayList;
import java.util.List;
import java.util.Map;
import java.util.TreeMap;
/**
* A {@link eu.binjr.core.data.adapters.DataAdapter} implementation capable of consuming data from the
* Netdata (https://netdata.cloud) API.
*
* @author Frederic Thevenet
*/
public class NetdataAdapter extends HttpDataAdapter<Double> {
private static final Logger logger = Logger.create(NetdataAdapter.class);
private static final char DELIMITER = ',';
private final Gson jsonParser;
private final ZoneId zoneId;
private final Decoder<Double> decoder;
private final UserPreferences userPrefs = UserPreferences.getInstance();
private final NetdataAdapterPreferences adapterPrefs = (NetdataAdapterPreferences) getAdapterInfo().getPreferences();
/**
* Initialises a new instance of the {@link NetdataAdapter} class.
*
* @throws CannotInitializeDataAdapterException if an error occurs while initializing the adapter.
*/
public NetdataAdapter() throws CannotInitializeDataAdapterException {
this(null, ZoneId.systemDefault());
}
private NetdataAdapter(URL baseAddress, ZoneId zoneId) throws CannotInitializeDataAdapterException {
super(baseAddress);
this.zoneId = zoneId;
this.decoder = buildDecoder(zoneId);
jsonParser = new Gson();
}
/**
* Returns a new instance of {@link NetdataAdapter} for the provided address and time zone.
*
* @param address the address of a Netdata server.
* @param zoneId the desired time zone.
* @return a new instance of {@link NetdataAdapter} for the provided address and time zone.
* @throws CannotInitializeDataAdapterException if an error occurs while initializing the adapter.
*/
public static DataAdapter<Double> fromUrl(String address, ZoneId zoneId) throws CannotInitializeDataAdapterException {
return new NetdataAdapter(urlFromString(address), zoneId);
}
@Override
protected URI craftFetchUri(String path, Instant begin, Instant end) throws DataAdapterException {
var params = new ArrayList<NameValuePair>();
params.add(UriParameter.of("points",
(userPrefs.downSamplingEnabled.get() && !adapterPrefs.disableServerSideDownsampling.get()
? userPrefs.downSamplingThreshold.get() : adapterPrefs.maxSamplesAllowed.get())));
params.add(UriParameter.of("group", adapterPrefs.groupingMethod.get()));
params.add(UriParameter.of("gtime", adapterPrefs.groupingTime.get()));
if (adapterPrefs.disableTimeFrameAlignment.get()) {
params.add(UriParameter.of("options", "unaligned"));
}
params.add(UriParameter.of("format", "csv"));
params.add(UriParameter.of("options", "seconds"));
params.add(UriParameter.of("after", begin.getEpochSecond() - adapterPrefs.fetchReadBehindSeconds.get().intValue()));
params.add(UriParameter.of("before", end.getEpochSecond() + adapterPrefs.fetchReadAheadSeconds.get().intValue()));
return craftRequestUri(path, params);
}
@Override
public boolean isSortingRequired() {
return true;
}
@Override
public Decoder<Double> getDecoder() {
return this.decoder;
}
@Override
public FilterableTreeItem<SourceBinding> getBindingTree() throws DataAdapterException {
var chartSummary = jsonParser.fromJson(doHttpGetJson(craftRequestUri(ChartSummary.ENDPOINT)), ChartSummary.class);
FilterableTreeItem<SourceBinding> tree = new FilterableTreeItem<>(
new TimeSeriesBinding.Builder()
.withAdapter(this)
.withLabel(getSourceName())
.withPath("/")
.build());
Map<String, FilterableTreeItem<SourceBinding>> types = new TreeMap<>();
chartSummary.getCharts().forEach((s, chart) -> {
var categoryName = getCategoryName(chart);
var categoryBranch = types.computeIfAbsent(categoryName, s1 -> new FilterableTreeItem<>(
new TimeSeriesBinding.Builder()
.withAdapter(this)
.withPath("")
.withLabel(categoryName)
.withParent(tree.getValue())
.build()));
var branch = new FilterableTreeItem<SourceBinding>(
new TimeSeriesBinding.Builder()
.withAdapter(this)
.withPath(chart.getDataUrl())
.withLabel(chart.getName())
.withGraphType(ChartType.valueOrDefault(chart.getChartType().name(), ChartType.STACKED))
.withLegend(chart.getTitle())
.withUnitName(chart.getUnits())
.withParent(categoryBranch.getValue())
.build());
chart.getDimensions().forEach((s1, chartDimensions) -> {
branch.getInternalChildren().add(0, new FilterableTreeItem<>(
new TimeSeriesBinding.Builder()
.withLabel(chartDimensions.getName())
.withAdapter(this)
.withParent(branch.getValue())
.withUnitName(chart.getUnits())
.withGraphType(ChartType.valueOrDefault(chart.getChartType().name(), ChartType.STACKED))
.withPath(chart.getDataUrl())
.build()));
});
categoryBranch.getInternalChildren().add(branch);
});
tree.getInternalChildren().addAll(types.values());
return tree;
}
@Override
public String getEncoding() {
return StandardCharsets.UTF_8.name();
}
@Override
public ZoneId getTimeZoneId() {
return zoneId;
}
@Override
public TimeRange getInitialTimeRange(String path, List<TimeSeriesInfo<Double>> seriesInfo) throws DataAdapterException {
Chart chart = jsonParser.fromJson(doHttpGetJson(craftRequestUri(path.replace("/data?", "/chart?"))), Chart.class);
return TimeRange.of(ZonedDateTime.ofInstant(Instant.ofEpochSecond(chart.getFirstEntry().longValue()), zoneId),
ZonedDateTime.ofInstant(Instant.ofEpochSecond(chart.getLastEntry().longValue()), zoneId));
}
@Override
public String getSourceName() {
return "[Netdata] " +
(getBaseAddress() != null ? getBaseAddress().getHost() : "???") +
((getBaseAddress() != null && getBaseAddress().getPort() > 0) ? ":" + getBaseAddress().getPort() : "") +
" - " +
" (" +
(zoneId != null ? zoneId : "???") +
")";
}
private CsvDecoder buildDecoder(ZoneId zoneId) {
return new CsvDecoder(getEncoding(), DELIMITER,
DoubleTimeSeriesProcessor::new,
s -> Instant.ofEpochSecond(Long.parseLong(s)).atZone(zoneId));
}
private String getCategoryName(Chart chart) {
var categoryName = chart.getType().split("_")[0];
return categoryName.isBlank() ?
"Unknown" : categoryName.substring(0, 1).toUpperCase() + categoryName.substring(1);
}
}
binjr/binjr-adapter-netdata/src/main/java/eu/binjr/sources/netdata/adapters/NetdataAdapterDialog.java
+49
View File
@@ -0,0 +1,49 @@
/*
* Copyright 2020 Frederic Thevenet
*
* Licensed under the Apache License, Version 2.0 (the "License");
* you may not use this file except in compliance with the License.
* You may obtain a copy of the License at
*
* http://www.apache.org/licenses/LICENSE-2.0
*
* Unless required by applicable law or agreed to in writing, software
* distributed under the License is distributed on an "AS IS" BASIS,
* WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
* See the License for the specific language governing permissions and
* limitations under the License.
*/
package eu.binjr.sources.netdata.adapters;
import eu.binjr.core.data.adapters.DataAdapter;
import eu.binjr.core.data.exceptions.DataAdapterException;
import eu.binjr.core.dialogs.DataAdapterDialog;
import javafx.scene.Node;
import java.net.URI;
import java.time.ZoneId;
import java.util.Collection;
import java.util.List;
/**
* A dialog box that returns a {@link NetdataAdapter} built according to user inputs.
*
* @author Frederic Thevenet
*/
public class NetdataAdapterDialog extends DataAdapterDialog<URI> {
/**
* Initializes a new instance of the {@link DataAdapterDialog} class.
*
* @param owner the owner window for the dialog
*/
public NetdataAdapterDialog(Node owner) {
super(owner, Mode.URI, "mostRecentNetdataUrls", false);
this.setDialogHeaderText("Connect to a Netdata source");
}
@Override
protected Collection<DataAdapter> getDataAdapters() throws DataAdapterException {
return List.of(NetdataAdapter.fromUrl(getSourceUri(), ZoneId.of(getSourceTimezone())));
}
}
binjr/binjr-adapter-netdata/src/main/java/eu/binjr/sources/netdata/adapters/NetdataAdapterInfo.java
+55
View File
@@ -0,0 +1,55 @@
/*
* Copyright 2020-2023 Frederic Thevenet
*
* Licensed under the Apache License, Version 2.0 (the "License");
* you may not use this file except in compliance with the License.
* You may obtain a copy of the License at
*
* http://www.apache.org/licenses/LICENSE-2.0
*
* Unless required by applicable law or agreed to in writing, software
* distributed under the License is distributed on an "AS IS" BASIS,
* WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
* See the License for the specific language governing permissions and
* limitations under the License.
*/
package eu.binjr.sources.netdata.adapters;
import eu.binjr.core.data.adapters.AdapterMetadata;
import eu.binjr.core.data.adapters.BaseDataAdapterInfo;
import eu.binjr.core.data.adapters.SourceLocality;
import eu.binjr.core.data.adapters.VisualizationType;
import eu.binjr.core.data.exceptions.CannotInitializeDataAdapterException;
import eu.binjr.core.data.exceptions.DataAdapterException;
import eu.binjr.core.preferences.AppEnvironment;
/**
* Defines the metadata associated with the NetdataDataAdapter
*
* @author Frederic Thevenet
*/
@AdapterMetadata(
name = "Netdata",
description = "Netdata Adapter",
copyright = AppEnvironment.COPYRIGHT_NOTICE,
license = AppEnvironment.LICENSE,
siteUrl = AppEnvironment.HTTP_WWW_BINJR_EU,
adapterClass = NetdataAdapter.class,
dialogClass = NetdataAdapterDialog.class,
preferencesClass = NetdataAdapterPreferences.class,
sourceLocality = SourceLocality.REMOTE,
apiLevel = AppEnvironment.PLUGIN_API_LEVEL,
visualizationType = VisualizationType.CHARTS
)
public class NetdataAdapterInfo extends BaseDataAdapterInfo {
/**
* Initialises a new instance of the {@link NetdataAdapterInfo} class.
*
* @throws CannotInitializeDataAdapterException if an error occurs initializing the adapter.
*/
public NetdataAdapterInfo() throws CannotInitializeDataAdapterException {
super(NetdataAdapterInfo.class);
}
}
binjr/binjr-adapter-netdata/src/main/java/eu/binjr/sources/netdata/adapters/NetdataAdapterPreferences.java
+77
View File
@@ -0,0 +1,77 @@
/*
* Copyright 2020 Frederic Thevenet
*
* Licensed under the Apache License, Version 2.0 (the "License");
* you may not use this file except in compliance with the License.
* You may obtain a copy of the License at
*
* http://www.apache.org/licenses/LICENSE-2.0
*
* Unless required by applicable law or agreed to in writing, software
* distributed under the License is distributed on an "AS IS" BASIS,
* WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
* See the License for the specific language governing permissions and
* limitations under the License.
*/
package eu.binjr.sources.netdata.adapters;
import eu.binjr.common.preferences.ObservablePreference;
import eu.binjr.core.data.adapters.DataAdapter;
import eu.binjr.core.data.adapters.DataAdapterPreferences;
import eu.binjr.sources.netdata.api.GroupingMethod;
/**
* Defines the preferences associated with the Netdata adapter.
*
* @author Frederic Thevenet
*/
public class NetdataAdapterPreferences extends DataAdapterPreferences {
/**
* Set to true to disable server-side down-sampling (aka "grouping"). Client-side down-sampling will still be applied.
*/
public ObservablePreference<Boolean> disableServerSideDownsampling = booleanPreference("disableServerSideDownsampling", false);
/**
* Set to true to disable Netdata alignment of all series on the same time-frame.
*/
public ObservablePreference<Boolean> disableTimeFrameAlignment = booleanPreference("disableTimeFrameAlignment", true);
/**
* Netdata's grouping (i.e. down-sampling) method: If multiple collected values are to be grouped in order to
* return fewer points, this parameters defines the method of grouping.
*/
public ObservablePreference<GroupingMethod> groupingMethod = enumPreference(GroupingMethod.class, "groupingMethod", GroupingMethod.AVERAGE);
/**
* The grouping number of seconds.
* This is used in conjunction with group=average to change the units of metrics
* (ie when the data is per-second, setting gtime=60 will turn them to per-minute).
*/
public ObservablePreference<Number> groupingTime = integerPreference("groupingTime", 0);
/**
* The amount of time in seconds to read after the specified time interval
*/
public final ObservablePreference<Number> fetchReadBehindSeconds = integerPreference("fetchReadBehindSeconds", 10);
/**
* The amount of time in seconds to read before the specified time interval
*/
public final ObservablePreference<Number> fetchReadAheadSeconds = integerPreference("fetchReadAheadSeconds", 10);
/**
* The maximum number of samples to recover from Netdata. 0 means every available samples will be returned.
*/
public ObservablePreference<Number> maxSamplesAllowed = integerPreference("maxSamplesAllowed", 10000);
/**
* Initialize a new instance of the {@link NetdataAdapterPreferences} class associated to
* a {@link DataAdapter} instance.
*
* @param dataAdapterClass the associated {@link DataAdapter}
*/
public NetdataAdapterPreferences(Class<? extends DataAdapter<?>> dataAdapterClass) {
super(dataAdapterClass);
}
}
binjr/binjr-adapter-netdata/src/main/java/eu/binjr/sources/netdata/api/Chart.java
+627
View File
@@ -0,0 +1,627 @@
/*
* NetData API
* Real-time performance and health monitoring.
*
* The version of the OpenAPI document: 1.11.1_rolling
*
*
* NOTE: This class is auto generated by OpenAPI Generator (https://openapi-generator.tech).
* https://openapi-generator.tech
* Do not edit the class manually.
*/
package eu.binjr.sources.netdata.api;
import com.google.gson.TypeAdapter;
import com.google.gson.annotations.JsonAdapter;
import com.google.gson.annotations.SerializedName;
import com.google.gson.stream.JsonReader;
import com.google.gson.stream.JsonWriter;
import java.io.IOException;
import java.math.BigDecimal;
import java.util.HashMap;
import java.util.Map;
import java.util.Objects;
/**
* Chart
*/
public class Chart {
public static final String SERIALIZED_NAME_ID = "id";
@SerializedName(SERIALIZED_NAME_ID)
private String id;
public static final String SERIALIZED_NAME_NAME = "name";
@SerializedName(SERIALIZED_NAME_NAME)
private String name;
public static final String SERIALIZED_NAME_TYPE = "type";
@SerializedName(SERIALIZED_NAME_TYPE)
private String type;
public static final String SERIALIZED_NAME_FAMILY = "family";
@SerializedName(SERIALIZED_NAME_FAMILY)
private String family;
public static final String SERIALIZED_NAME_TITLE = "title";
@SerializedName(SERIALIZED_NAME_TITLE)
private String title;
public static final String SERIALIZED_NAME_PRIORITY = "priority";
@SerializedName(SERIALIZED_NAME_PRIORITY)
private BigDecimal priority;
public static final String SERIALIZED_NAME_ENABLED = "enabled";
@SerializedName(SERIALIZED_NAME_ENABLED)
private Boolean enabled;
public static final String SERIALIZED_NAME_UNITS = "units";
@SerializedName(SERIALIZED_NAME_UNITS)
private String units;
public static final String SERIALIZED_NAME_DATA_URL = "data_url";
@SerializedName(SERIALIZED_NAME_DATA_URL)
private String dataUrl;
/**
* The chart type.
*/
@JsonAdapter(ChartTypeEnum.Adapter.class)
public enum ChartTypeEnum {
LINE("line"),
AREA("area"),
STACKED("stacked");
private String value;
ChartTypeEnum(String value) {
this.value = value;
}
public String getValue() {
return value;
}
@Override
public String toString() {
return String.valueOf(value);
}
public static ChartTypeEnum fromValue(String value) {
for (ChartTypeEnum b : ChartTypeEnum.values()) {
if (b.value.equals(value)) {
return b;
}
}
throw new IllegalArgumentException("Unexpected value '" + value + "'");
}
public static class Adapter extends TypeAdapter<ChartTypeEnum> {
@Override
public void write(final JsonWriter jsonWriter, final ChartTypeEnum enumeration) throws IOException {
jsonWriter.value(enumeration.getValue());
}
@Override
public ChartTypeEnum read(final JsonReader jsonReader) throws IOException {
String value = jsonReader.nextString();
return ChartTypeEnum.fromValue(value);
}
}
}
public static final String SERIALIZED_NAME_CHART_TYPE = "chart_type";
@SerializedName(SERIALIZED_NAME_CHART_TYPE)
private ChartTypeEnum chartType;
public static final String SERIALIZED_NAME_DURATION = "duration";
@SerializedName(SERIALIZED_NAME_DURATION)
private BigDecimal duration;
public static final String SERIALIZED_NAME_FIRST_ENTRY = "first_entry";
@SerializedName(SERIALIZED_NAME_FIRST_ENTRY)
private BigDecimal firstEntry;
public static final String SERIALIZED_NAME_LAST_ENTRY = "last_entry";
@SerializedName(SERIALIZED_NAME_LAST_ENTRY)
private BigDecimal lastEntry;
public static final String SERIALIZED_NAME_UPDATE_EVERY = "update_every";
@SerializedName(SERIALIZED_NAME_UPDATE_EVERY)
private BigDecimal updateEvery;
public static final String SERIALIZED_NAME_DIMENSIONS = "dimensions";
@SerializedName(SERIALIZED_NAME_DIMENSIONS)
private Map<String, ChartDimensions> dimensions = null;
// public static final String SERIALIZED_NAME_CHART_VARIABLES = "chart_variables";
// @SerializedName(SERIALIZED_NAME_CHART_VARIABLES)
// private Map<String, ChartVariables> chartVariables = null;
public static final String SERIALIZED_NAME_GREEN = "green";
@SerializedName(SERIALIZED_NAME_GREEN)
private BigDecimal green;
public static final String SERIALIZED_NAME_RED = "red";
@SerializedName(SERIALIZED_NAME_RED)
private BigDecimal red;
public Chart id(String id) {
this.id = id;
return this;
}
/**
* The unique id of the chart.
*
* @return id
**/
public String getId() {
return id;
}
public void setId(String id) {
this.id = id;
}
public Chart name(String name) {
this.name = name;
return this;
}
/**
* The name of the chart.
*
* @return name
**/
public String getName() {
return name;
}
public void setName(String name) {
this.name = name;
}
public Chart type(String type) {
this.type = type;
return this;
}
/**
* The type of the chart. Types are not handled by netdata. You can use this field for anything you like.
*
* @return type
**/
public String getType() {
return type;
}
public void setType(String type) {
this.type = type;
}
public Chart family(String family) {
this.family = family;
return this;
}
/**
* The family of the chart. Families are not handled by netdata. You can use this field for anything you like.
*
* @return family
**/
public String getFamily() {
return family;
}
public void setFamily(String family) {
this.family = family;
}
public Chart title(String title) {
this.title = title;
return this;
}
/**
* The title of the chart.
*
* @return title
**/
public String getTitle() {
return title;
}
public void setTitle(String title) {
this.title = title;
}
public Chart priority(BigDecimal priority) {
this.priority = priority;
return this;
}
/**
* The relative priority of the chart. NetData does not care about priorities. This is just an indication of importance for the chart viewers to sort charts of higher priority (lower number) closer to the top. Priority sorting should only be used among charts of the same type or family.
*
* @return priority
**/
public BigDecimal getPriority() {
return priority;
}
public void setPriority(BigDecimal priority) {
this.priority = priority;
}
public Chart enabled(Boolean enabled) {
this.enabled = enabled;
return this;
}
/**
* True when the chart is enabled. Disabled charts do not currently collect values, but they may have historical values available.
*
* @return enabled
**/
public Boolean getEnabled() {
return enabled;
}
public void setEnabled(Boolean enabled) {
this.enabled = enabled;
}
public Chart units(String units) {
this.units = units;
return this;
}
/**
* The unit of measurement for the values of all dimensions of the chart.
*
* @return units
**/
public String getUnits() {
return units;
}
public void setUnits(String units) {
this.units = units;
}
public Chart dataUrl(String dataUrl) {
this.dataUrl = dataUrl;
return this;
}
/**
* The absolute path to get data values for this chart. You are expected to use this path as the base when constructing the URL to fetch data values for this chart.
*
* @return dataUrl
**/
public String getDataUrl() {
return dataUrl;
}
public void setDataUrl(String dataUrl) {
this.dataUrl = dataUrl;
}
public Chart chartType(ChartTypeEnum chartType) {
this.chartType = chartType;
return this;
}
/**
* The chart type.
*
* @return chartType
**/
public ChartTypeEnum getChartType() {
return chartType;
}
public void setChartType(ChartTypeEnum chartType) {
this.chartType = chartType;
}
public Chart duration(BigDecimal duration) {
this.duration = duration;
return this;
}
/**
* The duration, in seconds, of the round robin database maintained by netdata.
*
* @return duration
**/
public BigDecimal getDuration() {
return duration;
}
public void setDuration(BigDecimal duration) {
this.duration = duration;
}
public Chart firstEntry(BigDecimal firstEntry) {
this.firstEntry = firstEntry;
return this;
}
/**
* The UNIX timestamp of the first entry (the oldest) in the round robin database.
*
* @return firstEntry
**/
public BigDecimal getFirstEntry() {
return firstEntry;
}
public void setFirstEntry(BigDecimal firstEntry) {
this.firstEntry = firstEntry;
}
public Chart lastEntry(BigDecimal lastEntry) {
this.lastEntry = lastEntry;
return this;
}
/**
* The UNIX timestamp of the latest entry in the round robin database.
*
* @return lastEntry
**/
public BigDecimal getLastEntry() {
return lastEntry;
}
public void setLastEntry(BigDecimal lastEntry) {
this.lastEntry = lastEntry;
}
public Chart updateEvery(BigDecimal updateEvery) {
this.updateEvery = updateEvery;
return this;
}
/**
* The update frequency of this chart, in seconds. One value every this amount of time is kept in the round robin database.
*
* @return updateEvery
**/
public BigDecimal getUpdateEvery() {
return updateEvery;
}
public void setUpdateEvery(BigDecimal updateEvery) {
this.updateEvery = updateEvery;
}
public Chart dimensions(Map<String, ChartDimensions> dimensions) {
this.dimensions = dimensions;
return this;
}
public Chart putDimensionsItem(String key, ChartDimensions dimensionsItem) {
if (this.dimensions == null) {
this.dimensions = new HashMap<>();
}
this.dimensions.put(key, dimensionsItem);
return this;
}
/**
* An object containing all the chart dimensions available for the chart. This is used as an indexed array. For each pair in the dictionary: the key is the id of the dimension and the value is a dictionary containing the name.
*
* @return dimensions
**/
public Map<String, ChartDimensions> getDimensions() {
return dimensions;
}
public void setDimensions(Map<String, ChartDimensions> dimensions) {
this.dimensions = dimensions;
}
// public Chart chartVariables(Map<String, ChartVariables> chartVariables) {
//
// this.chartVariables = chartVariables;
// return this;
// }
// public Chart putChartVariablesItem(String key, ChartVariables chartVariablesItem) {
// if (this.chartVariables == null) {
// this.chartVariables = new HashMap<>();
// }
// this.chartVariables.put(key, chartVariablesItem);
// return this;
// }
//
// /**
// * Get chartVariables
// *
// * @return chartVariables
// **/
//
//
// public Map<String, ChartVariables> getChartVariables() {
// return chartVariables;
// }
//
//
// public void setChartVariables(Map<String, ChartVariables> chartVariables) {
// this.chartVariables = chartVariables;
// }
public Chart green(BigDecimal green) {
this.green = green;
return this;
}
/**
* Chart health green threshold.
*
* @return green
**/
public BigDecimal getGreen() {
return green;
}
public void setGreen(BigDecimal green) {
this.green = green;
}
public Chart red(BigDecimal red) {
this.red = red;
return this;
}
/**
* Chart health red threshold.
*
* @return red
**/
public BigDecimal getRed() {
return red;
}
public void setRed(BigDecimal red) {
this.red = red;
}
@Override
public boolean equals(Object o) {
if (this == o) {
return true;
}
if (o == null || getClass() != o.getClass()) {
return false;
}
Chart chart = (Chart) o;
return Objects.equals(this.id, chart.id) &&
Objects.equals(this.name, chart.name) &&
Objects.equals(this.type, chart.type) &&
Objects.equals(this.family, chart.family) &&
Objects.equals(this.title, chart.title) &&
Objects.equals(this.priority, chart.priority) &&
Objects.equals(this.enabled, chart.enabled) &&
Objects.equals(this.units, chart.units) &&
Objects.equals(this.dataUrl, chart.dataUrl) &&
Objects.equals(this.chartType, chart.chartType) &&
Objects.equals(this.duration, chart.duration) &&
Objects.equals(this.firstEntry, chart.firstEntry) &&
Objects.equals(this.lastEntry, chart.lastEntry) &&
Objects.equals(this.updateEvery, chart.updateEvery) &&
Objects.equals(this.dimensions, chart.dimensions) &&
// Objects.equals(this.chartVariables, chart.chartVariables) &&
Objects.equals(this.green, chart.green) &&
Objects.equals(this.red, chart.red);
}
@Override
public int hashCode() {
return Objects.hash(id, name, type, family, title, priority, enabled, units, dataUrl, chartType, duration, firstEntry, lastEntry, updateEvery, dimensions, /*chartVariables,*/ green, red);
}
@Override
public String toString() {
StringBuilder sb = new StringBuilder();
sb.append("class Chart {\n");
sb.append(" id: ").append(toIndentedString(id)).append("\n");
sb.append(" name: ").append(toIndentedString(name)).append("\n");
sb.append(" type: ").append(toIndentedString(type)).append("\n");
sb.append(" family: ").append(toIndentedString(family)).append("\n");
sb.append(" title: ").append(toIndentedString(title)).append("\n");
sb.append(" priority: ").append(toIndentedString(priority)).append("\n");
sb.append(" enabled: ").append(toIndentedString(enabled)).append("\n");
sb.append(" units: ").append(toIndentedString(units)).append("\n");
sb.append(" dataUrl: ").append(toIndentedString(dataUrl)).append("\n");
sb.append(" chartType: ").append(toIndentedString(chartType)).append("\n");
sb.append(" duration: ").append(toIndentedString(duration)).append("\n");
sb.append(" firstEntry: ").append(toIndentedString(firstEntry)).append("\n");
sb.append(" lastEntry: ").append(toIndentedString(lastEntry)).append("\n");
sb.append(" updateEvery: ").append(toIndentedString(updateEvery)).append("\n");
sb.append(" dimensions: ").append(toIndentedString(dimensions)).append("\n");
// sb.append(" chartVariables: ").append(toIndentedString(chartVariables)).append("\n");
sb.append(" green: ").append(toIndentedString(green)).append("\n");
sb.append(" red: ").append(toIndentedString(red)).append("\n");
sb.append("}");
return sb.toString();
}
/**
* Convert the given object to string with each line indented by 4 spaces
* (except the first line).
*/
private String toIndentedString(Object o) {
if (o == null) {
return "null";
}
return o.toString().replace("\n", "\n ");
}
}
binjr/binjr-adapter-netdata/src/main/java/eu/binjr/sources/netdata/api/ChartDimensions.java
+89
View File
@@ -0,0 +1,89 @@
/*
* NetData API
* Real-time performance and health monitoring.
*
* The version of the OpenAPI document: 1.11.1_rolling
*
*
* NOTE: This class is auto generated by OpenAPI Generator (https://openapi-generator.tech).
* https://openapi-generator.tech
* Do not edit the class manually.
*/
package eu.binjr.sources.netdata.api;
import com.google.gson.annotations.SerializedName;
import java.util.Objects;
/**
* ChartDimensions
*/
public class ChartDimensions {
public static final String SERIALIZED_NAME_NAME = "name";
@SerializedName(SERIALIZED_NAME_NAME)
private String name;
public ChartDimensions name(String name) {
this.name = name;
return this;
}
/**
* The name of the dimension
* @return name
**/
public String getName() {
return name;
}
public void setName(String name) {
this.name = name;
}
@Override
public boolean equals(Object o) {
if (this == o) {
return true;
}
if (o == null || getClass() != o.getClass()) {
return false;
}
ChartDimensions chartDimensions = (ChartDimensions) o;
return Objects.equals(this.name, chartDimensions.name);
}
@Override
public int hashCode() {
return Objects.hash(name);
}
@Override
public String toString() {
StringBuilder sb = new StringBuilder();
sb.append("class ChartDimensions {\n");
sb.append(" name: ").append(toIndentedString(name)).append("\n");
sb.append("}");
return sb.toString();
}
/**
* Convert the given object to string with each line indented by 4 spaces
* (except the first line).
*/
private String toIndentedString(Object o) {
if (o == null) {
return "null";
}
return o.toString().replace("\n", "\n ");
}
}
binjr/binjr-adapter-netdata/src/main/java/eu/binjr/sources/netdata/api/ChartSummary.java
+475
View File
@@ -0,0 +1,475 @@
/*
* NetData API
* Real-time performance and health monitoring.
*
* The version of the OpenAPI document: 1.11.1_rolling
*
*
* NOTE: This class is auto generated by OpenAPI Generator (https://openapi-generator.tech).
* https://openapi-generator.tech
* Do not edit the class manually.
*/
package eu.binjr.sources.netdata.api;
import com.google.gson.TypeAdapter;
import com.google.gson.annotations.JsonAdapter;
import com.google.gson.annotations.SerializedName;
import com.google.gson.stream.JsonReader;
import com.google.gson.stream.JsonWriter;
import java.io.IOException;
import java.math.BigDecimal;
import java.util.HashMap;
import java.util.Map;
import java.util.Objects;
/**
* ChartSummary
*/
public class ChartSummary {
public static final String ENDPOINT= "/api/v1/charts";
public static final String SERIALIZED_NAME_HOSTNAME = "hostname";
@SerializedName(SERIALIZED_NAME_HOSTNAME)
private String hostname;
public static final String SERIALIZED_NAME_VERSION = "version";
@SerializedName(SERIALIZED_NAME_VERSION)
private String version;
public static final String SERIALIZED_NAME_RELEASE_CHANNEL = "release_channel";
@SerializedName(SERIALIZED_NAME_RELEASE_CHANNEL)
private String releaseChannel;
public static final String SERIALIZED_NAME_TIMEZONE = "timezone";
@SerializedName(SERIALIZED_NAME_TIMEZONE)
private String timezone;
/**
* The netdata server host operating system.
*/
@JsonAdapter(OsEnum.Adapter.class)
public enum OsEnum {
MACOS("macos"),
LINUX("linux"),
FREEBSD("freebsd");
private String value;
OsEnum(String value) {
this.value = value;
}
public String getValue() {
return value;
}
@Override
public String toString() {
return String.valueOf(value);
}
public static OsEnum fromValue(String value) {
for (OsEnum b : OsEnum.values()) {
if (b.value.equals(value)) {
return b;
}
}
throw new IllegalArgumentException("Unexpected value '" + value + "'");
}
public static class Adapter extends TypeAdapter<OsEnum> {
@Override
public void write(final JsonWriter jsonWriter, final OsEnum enumeration) throws IOException {
jsonWriter.value(enumeration.getValue());
}
@Override
public OsEnum read(final JsonReader jsonReader) throws IOException {
String value = jsonReader.nextString();
return OsEnum.fromValue(value);
}
}
}
public static final String SERIALIZED_NAME_OS = "os";
@SerializedName(SERIALIZED_NAME_OS)
private OsEnum os;
public static final String SERIALIZED_NAME_HISTORY = "history";
@SerializedName(SERIALIZED_NAME_HISTORY)
private BigDecimal history;
public static final String SERIALIZED_NAME_MEMORY_MODE = "memory_mode";
@SerializedName(SERIALIZED_NAME_MEMORY_MODE)
private String memoryMode;
public static final String SERIALIZED_NAME_UPDATE_EVERY = "update_every";
@SerializedName(SERIALIZED_NAME_UPDATE_EVERY)
private BigDecimal updateEvery;
public static final String SERIALIZED_NAME_CHARTS = "charts";
@SerializedName(SERIALIZED_NAME_CHARTS)
private Map<String, Chart> charts = null;
public static final String SERIALIZED_NAME_CHARTS_COUNT = "charts_count";
@SerializedName(SERIALIZED_NAME_CHARTS_COUNT)
private BigDecimal chartsCount;
public static final String SERIALIZED_NAME_DIMENSIONS_COUNT = "dimensions_count";
@SerializedName(SERIALIZED_NAME_DIMENSIONS_COUNT)
private BigDecimal dimensionsCount;
public static final String SERIALIZED_NAME_ALARMS_COUNT = "alarms_count";
@SerializedName(SERIALIZED_NAME_ALARMS_COUNT)
private BigDecimal alarmsCount;
public static final String SERIALIZED_NAME_RRD_MEMORY_BYTES = "rrd_memory_bytes";
@SerializedName(SERIALIZED_NAME_RRD_MEMORY_BYTES)
private BigDecimal rrdMemoryBytes;
public ChartSummary hostname(String hostname) {
this.hostname = hostname;
return this;
}
/**
* The hostname of the netdata server.
*
* @return hostname
**/
public String getHostname() {
return hostname;
}
public void setHostname(String hostname) {
this.hostname = hostname;
}
public ChartSummary version(String version) {
this.version = version;
return this;
}
/**
* netdata version of the server.
*
* @return version
**/
public String getVersion() {
return version;
}
public void setVersion(String version) {
this.version = version;
}
public ChartSummary releaseChannel(String releaseChannel) {
this.releaseChannel = releaseChannel;
return this;
}
/**
* The release channel of the build on the server.
*
* @return releaseChannel
**/
public String getReleaseChannel() {
return releaseChannel;
}
public void setReleaseChannel(String releaseChannel) {
this.releaseChannel = releaseChannel;
}
public ChartSummary timezone(String timezone) {
this.timezone = timezone;
return this;
}
/**
* The current timezone on the server.
*
* @return timezone
**/
public String getTimezone() {
return timezone;
}
public void setTimezone(String timezone) {
this.timezone = timezone;
}
public ChartSummary os(OsEnum os) {
this.os = os;
return this;
}
/**
* The netdata server host operating system.
*
* @return os
**/
public OsEnum getOs() {
return os;
}
public void setOs(OsEnum os) {
this.os = os;
}
public ChartSummary history(BigDecimal history) {
this.history = history;
return this;
}
/**
* The duration, in seconds, of the round robin database maintained by netdata.
*
* @return history
**/
public BigDecimal getHistory() {
return history;
}
public void setHistory(BigDecimal history) {
this.history = history;
}
public ChartSummary memoryMode(String memoryMode) {
this.memoryMode = memoryMode;
return this;
}
/**
* The name of the database memory mode on the server.
*
* @return memoryMode
**/
public String getMemoryMode() {
return memoryMode;
}
public void setMemoryMode(String memoryMode) {
this.memoryMode = memoryMode;
}
public ChartSummary updateEvery(BigDecimal updateEvery) {
this.updateEvery = updateEvery;
return this;
}
/**
* The default update frequency of the netdata server. All charts have an update frequency equal or bigger than this.
*
* @return updateEvery
**/
public BigDecimal getUpdateEvery() {
return updateEvery;
}
public void setUpdateEvery(BigDecimal updateEvery) {
this.updateEvery = updateEvery;
}
public ChartSummary charts(Map<String, Chart> charts) {
this.charts = charts;
return this;
}
public ChartSummary putChartsItem(String key, Chart chartsItem) {
if (this.charts == null) {
this.charts = new HashMap<>();
}
this.charts.put(key, chartsItem);
return this;
}
/**
* An object containing all the chart objects available at the netdata server. This is used as an indexed array. The key of each chart object is the id of the chart.
*
* @return charts
**/
public Map<String, Chart> getCharts() {
return charts;
}
public void setCharts(Map<String, Chart> charts) {
this.charts = charts;
}
public ChartSummary chartsCount(BigDecimal chartsCount) {
this.chartsCount = chartsCount;
return this;
}
/**
* The number of charts.
*
* @return chartsCount
**/
public BigDecimal getChartsCount() {
return chartsCount;
}
public void setChartsCount(BigDecimal chartsCount) {
this.chartsCount = chartsCount;
}
public ChartSummary dimensionsCount(BigDecimal dimensionsCount) {
this.dimensionsCount = dimensionsCount;
return this;
}
/**
* The total number of dimensions.
*
* @return dimensionsCount
**/
public BigDecimal getDimensionsCount() {
return dimensionsCount;
}
public void setDimensionsCount(BigDecimal dimensionsCount) {
this.dimensionsCount = dimensionsCount;
}
public ChartSummary alarmsCount(BigDecimal alarmsCount) {
this.alarmsCount = alarmsCount;
return this;
}
/**
* The number of alarms.
*
* @return alarmsCount
**/
public BigDecimal getAlarmsCount() {
return alarmsCount;
}
public void setAlarmsCount(BigDecimal alarmsCount) {
this.alarmsCount = alarmsCount;
}
public ChartSummary rrdMemoryBytes(BigDecimal rrdMemoryBytes) {
this.rrdMemoryBytes = rrdMemoryBytes;
return this;
}
/**
* The size of the round robin database in bytes.
*
* @return rrdMemoryBytes
**/
public BigDecimal getRrdMemoryBytes() {
return rrdMemoryBytes;
}
public void setRrdMemoryBytes(BigDecimal rrdMemoryBytes) {
this.rrdMemoryBytes = rrdMemoryBytes;
}
@Override
public boolean equals(Object o) {
if (this == o) {
return true;
}
if (o == null || getClass() != o.getClass()) {
return false;
}
ChartSummary chartSummary = (ChartSummary) o;
return Objects.equals(this.hostname, chartSummary.hostname) &&
Objects.equals(this.version, chartSummary.version) &&
Objects.equals(this.releaseChannel, chartSummary.releaseChannel) &&
Objects.equals(this.timezone, chartSummary.timezone) &&
Objects.equals(this.os, chartSummary.os) &&
Objects.equals(this.history, chartSummary.history) &&
Objects.equals(this.memoryMode, chartSummary.memoryMode) &&
Objects.equals(this.updateEvery, chartSummary.updateEvery) &&
Objects.equals(this.charts, chartSummary.charts) &&
Objects.equals(this.chartsCount, chartSummary.chartsCount) &&
Objects.equals(this.dimensionsCount, chartSummary.dimensionsCount) &&
Objects.equals(this.alarmsCount, chartSummary.alarmsCount) &&
Objects.equals(this.rrdMemoryBytes, chartSummary.rrdMemoryBytes);
}
@Override
public int hashCode() {
return Objects.hash(hostname, version, releaseChannel, timezone, os, history, memoryMode, updateEvery, charts, chartsCount, dimensionsCount, alarmsCount, rrdMemoryBytes);
}
@Override
public String toString() {
StringBuilder sb = new StringBuilder();
sb.append("class ChartSummary {\n");
sb.append(" hostname: ").append(toIndentedString(hostname)).append("\n");
sb.append(" version: ").append(toIndentedString(version)).append("\n");
sb.append(" releaseChannel: ").append(toIndentedString(releaseChannel)).append("\n");
sb.append(" timezone: ").append(toIndentedString(timezone)).append("\n");
sb.append(" os: ").append(toIndentedString(os)).append("\n");
sb.append(" history: ").append(toIndentedString(history)).append("\n");
sb.append(" memoryMode: ").append(toIndentedString(memoryMode)).append("\n");
sb.append(" updateEvery: ").append(toIndentedString(updateEvery)).append("\n");
sb.append(" charts: ").append(toIndentedString(charts)).append("\n");
sb.append(" chartsCount: ").append(toIndentedString(chartsCount)).append("\n");
sb.append(" dimensionsCount: ").append(toIndentedString(dimensionsCount)).append("\n");
sb.append(" alarmsCount: ").append(toIndentedString(alarmsCount)).append("\n");
sb.append(" rrdMemoryBytes: ").append(toIndentedString(rrdMemoryBytes)).append("\n");
sb.append("}");
return sb.toString();
}
/**
* Convert the given object to string with each line indented by 4 spaces
* (except the first line).
*/
private String toIndentedString(Object o) {
if (o == null) {
return "null";
}
return o.toString().replace("\n", "\n ");
}
}
binjr/binjr-adapter-netdata/src/main/java/eu/binjr/sources/netdata/api/GroupingMethod.java
+41
View File
@@ -0,0 +1,41 @@
/*
* Copyright 2020 Frederic Thevenet
*
* Licensed under the Apache License, Version 2.0 (the "License");
* you may not use this file except in compliance with the License.
* You may obtain a copy of the License at
*
* http://www.apache.org/licenses/LICENSE-2.0
*
* Unless required by applicable law or agreed to in writing, software
* distributed under the License is distributed on an "AS IS" BASIS,
* WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
* See the License for the specific language governing permissions and
* limitations under the License.
*/
package eu.binjr.sources.netdata.api;
/**
* The gouping methods supported by Netdata
*/
public enum GroupingMethod {
MIN("min"),
MAX("max"),
AVERAGE("average"),
SUM("sum"),
MEDIAN("median"),
STDDEV("stddev"),
INCREMENTAL_SUM("incremental-sum");
private final String parameterValue;
GroupingMethod(String parameterValue) {
this.parameterValue = parameterValue;
}
@Override
public String toString() {
return this.parameterValue;
}
}
binjr/binjr-adapter-netdata/src/main/resources/META-INF/services/eu.binjr.core.data.adapters.DataAdapterInfo
+18
View File
@@ -0,0 +1,18 @@
#
# Copyright 2020 Frederic Thevenet
#
# Licensed under the Apache License, Version 2.0 (the "License");
# you may not use this file except in compliance with the License.
# You may obtain a copy of the License at
#
# http://www.apache.org/licenses/LICENSE-2.0
#
# Unless required by applicable law or agreed to in writing, software
# distributed under the License is distributed on an "AS IS" BASIS,
# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
# See the License for the specific language governing permissions and
# limitations under the License.
#
# Netdata Data adapter service implementation
eu.binjr.sources.netdata.adapters.NetdataAdapterInfo
binjr/binjr-adapter-rrd4j/README.md
+10
View File
@@ -0,0 +1,10 @@
# binjr-adapter-rrd4j
[![Maven Central](https://img.shields.io/maven-central/v/eu.binjr/binjr-adapter-rrd4j.svg?label=Maven%20Central&style=flat-square)](https://search.maven.org/search?q=g:%22eu.binjr%22%20AND%20a:%22binjr-adapter-rrd4j%22)
This module implements a DataAdapter capable of consuming data from Round Robin Database files.
It can read files produced by the following applications:
* [RRD4J](https://github.com/rrd4j/rrd4j), binary files (*.rrd) and XML dumps (*.xml)
* [RRDTool](https://oss.oetiker.ch/rrdtool/index.en.html), binary files (*.rrd) and XML dumps (*.xml)
binjr/binjr-adapter-rrd4j/build.gradle
+32
View File
@@ -0,0 +1,32 @@
/*
* Copyright 2018-2022 Frederic Thevenet
*
* Licensed under the Apache License, Version 2.0 (the "License");
* you may not use this file except in compliance with the License.
* You may obtain a copy of the License at
*
* http://www.apache.org/licenses/LICENSE-2.0
*
* Unless required by applicable law or agreed to in writing, software
* distributed under the License is distributed on an "AS IS" BASIS,
* WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
* See the License for the specific language governing permissions and
* limitations under the License.
*/
dependencies {
compileOnly project(':binjr-core')
implementation 'org.rrd4j:rrd4j:3.9'
}
jar {
manifest {
attributes(
'Specification-Title': project.name,
'Specification-Version': project.version,
'Implementation-Title': project.name,
'Implementation-Version': project.version,
'Build-Number': BINJR_BUILD_NUMBER
)
}
}
binjr/binjr-adapter-rrd4j/src/main/java/eu/binjr/sources/rrd4j/adapters/Rrd4jBackendType.java
+40
View File
@@ -0,0 +1,40 @@
/*
* Copyright 2020 Frederic Thevenet
*
* Licensed under the Apache License, Version 2.0 (the "License");
* you may not use this file except in compliance with the License.
* You may obtain a copy of the License at
*
* http://www.apache.org/licenses/LICENSE-2.0
*
* Unless required by applicable law or agreed to in writing, software
* distributed under the License is distributed on an "AS IS" BASIS,
* WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
* See the License for the specific language governing permissions and
* limitations under the License.
*/
package eu.binjr.sources.rrd4j.adapters;
/**
* RRD4J backend types.
*/
public enum Rrd4jBackendType {
/**
* Backend based on the java.io.* package. RRD data is stored in files on the disk
*/
FILE,
/**
* Backend based on the java.io.* package. RRD data is stored in files on the disk.
* This backend is "safe". Being safe means that RRD files can be safely shared between several JVM's.
*/
SAFE,
/**
* Backend based on the java.nio.* package. RRD data is stored in files on the disk
*/
NIO,
/**
* Memory-oriented backend. RRD data is stored in memory, it gets lost as soon as JVM exits.
*/
MEMORY;
}
binjr/binjr-adapter-rrd4j/src/main/java/eu/binjr/sources/rrd4j/adapters/Rrd4jFileAdapter.java
+272
View File
@@ -0,0 +1,272 @@
/*
* Copyright 2018-2022 Frederic Thevenet
*
* Licensed under the Apache License, Version 2.0 (the "License");
* you may not use this file except in compliance with the License.
* You may obtain a copy of the License at
*
* http://www.apache.org/licenses/LICENSE-2.0
*
* Unless required by applicable law or agreed to in writing, software
* distributed under the License is distributed on an "AS IS" BASIS,
* WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
* See the License for the specific language governing permissions and
* limitations under the License.
*/
package eu.binjr.sources.rrd4j.adapters;
import eu.binjr.common.javafx.controls.TimeRange;
import eu.binjr.common.logging.Logger;
import eu.binjr.core.data.adapters.BaseDataAdapter;
import eu.binjr.core.data.adapters.SourceBinding;
import eu.binjr.core.data.adapters.TimeSeriesBinding;
import eu.binjr.core.data.exceptions.DataAdapterException;
import eu.binjr.core.data.exceptions.FetchingDataFromAdapterException;
import eu.binjr.core.data.timeseries.DoubleTimeSeriesProcessor;
import eu.binjr.core.data.timeseries.TimeSeriesProcessor;
import eu.binjr.core.data.workspace.TimeSeriesInfo;
import eu.binjr.core.preferences.UserPreferences;
import javafx.scene.chart.XYChart;
import javafx.scene.control.TreeItem;
import org.eclipse.fx.ui.controls.tree.FilterableTreeItem;
import org.rrd4j.ConsolFun;
import org.rrd4j.core.*;
import java.io.IOException;
import java.nio.file.Files;
import java.nio.file.Path;
import java.nio.file.Paths;
import java.time.Instant;
import java.time.ZoneId;
import java.time.ZonedDateTime;
import java.util.*;
import java.util.stream.Collectors;
/**
* A {@link eu.binjr.core.data.adapters.DataAdapter} implementation capable of consuming data
* from Round Robin Database files.
*
* @author Frederic Thevenet
*/
public class Rrd4jFileAdapter extends BaseDataAdapter<Double> {
private static final Logger logger = Logger.create(Rrd4jFileAdapter.class);
private final Rrd4jFileAdapterPreferences prefs = (Rrd4jFileAdapterPreferences) this.getAdapterInfo().getPreferences();
private final Map<Path, RrdDb> rrdDbMap = new HashMap<>();
private List<Path> rrdPaths;
private List<Path> tempPathToCollect = new ArrayList<>();
/**
* Initialises a new instance of the {@link Rrd4jFileAdapter} class.
*/
public Rrd4jFileAdapter() {
this(new ArrayList<Path>());
}
/**
* Initialises a new instance of the {@link Rrd4jFileAdapter} class from the provided list of {@link Path}
*
* @param rrdPath a list of {@link Path} to be mounted by the adapter.
*/
public Rrd4jFileAdapter(List<Path> rrdPath) {
this.rrdPaths = rrdPath;
}
@Override
public FilterableTreeItem<SourceBinding> getBindingTree() throws DataAdapterException {
FilterableTreeItem<SourceBinding> tree = new FilterableTreeItem<>(
new TimeSeriesBinding.Builder()
.withLabel(getSourceName())
.withPath("/")
.withAdapter(this)
.build());
for (Path rrdPath : rrdPaths) {
try {
String rrdFileName = rrdPath.getFileName().toString();
FilterableTreeItem<SourceBinding> rrdNode = new FilterableTreeItem<>(
new TimeSeriesBinding.Builder()
.withLabel(rrdFileName)
.withPath(rrdFileName)
.withParent(tree.getValue())
.withAdapter(this)
.build());
RrdDb rrd = openRrdDb(rrdPath);
rrdDbMap.put(rrdPath, rrd);
for (ConsolFun consolFun : Arrays.stream(rrd.getRrdDef().getArcDefs())
.map(ArcDef::getConsolFun)
.collect(Collectors.toSet())) {
FilterableTreeItem<SourceBinding> consolFunNode = new FilterableTreeItem<>(new TimeSeriesBinding.Builder()
.withLabel(consolFun.toString())
.withPath(rrdPath.resolve(consolFun.toString()).toString())
.withParent(rrdNode.getValue())
.withAdapter(this)
.build());
rrdNode.getInternalChildren().add(consolFunNode);
for (String ds : rrd.getDsNames()) {
consolFunNode.getInternalChildren().add(new TreeItem<>(new TimeSeriesBinding.Builder()
.withLabel(ds)
.withPath(consolFunNode.getValue().getPath())
.withParent(consolFunNode.getValue())
.withAdapter(this)
.build()));
}
}
tree.getInternalChildren().add(rrdNode);
} catch (IOException e) {
throw new DataAdapterException("Failed to open rrd db", e);
}
}
return tree;
}
@Override
public TimeRange getInitialTimeRange(String path, List<TimeSeriesInfo<Double>> seriesInfo) throws DataAdapterException {
if (this.isClosed()) {
throw new IllegalStateException("An attempt was made to fetch data from a closed adapter");
}
Path dsPath = Path.of(path);
try {
var end = Instant.ofEpochSecond(rrdDbMap.get(dsPath.getParent()).getLastArchiveUpdateTime()).atZone(getTimeZoneId());
return TimeRange.of(end.minusHours(24), end);
} catch (IOException e) {
throw new FetchingDataFromAdapterException("IO Error while retrieving last update from rrd db", e);
}
}
@Override
public Map<TimeSeriesInfo<Double>, TimeSeriesProcessor<Double>> fetchData(String path, Instant begin, Instant
end, List<TimeSeriesInfo<Double>> seriesInfo, boolean bypassCache)
throws DataAdapterException {
if (this.isClosed()) {
throw new IllegalStateException("An attempt was made to fetch data from a closed adapter");
}
Path dsPath = Path.of(path);
try {
FetchRequest request = rrdDbMap.get(dsPath.getParent()).createFetchRequest(
ConsolFun.valueOf(dsPath.getFileName().toString()),
begin.getEpochSecond(),
end.getEpochSecond());
request.setFilter(seriesInfo.stream().map(s -> s.getBinding().getLabel()).toArray(String[]::new));
FetchData data = request.fetchData();
Map<TimeSeriesInfo<Double>, TimeSeriesProcessor<Double>> series = new HashMap<>();
for (int i = 0; i < data.getRowCount(); i++) {
ZonedDateTime timeStamp = Instant.ofEpochSecond(data.getTimestamps()[i]).atZone(getTimeZoneId());
for (TimeSeriesInfo<Double> info : seriesInfo) {
Double val = data.getValues(info.getBinding().getLabel())[i];
XYChart.Data<ZonedDateTime, Double> point = new XYChart.Data<>(timeStamp, val);
TimeSeriesProcessor<Double> seriesProcessor =
series.computeIfAbsent(info, k -> new DoubleTimeSeriesProcessor());
seriesProcessor.addSample(point);
}
}
logger.trace(() -> String.format("Built %d series with %d samples each (%d total samples)",
seriesInfo.size(),
data.getRowCount(),
seriesInfo.size() * data.getRowCount()));
return series;
} catch (IOException e) {
throw new FetchingDataFromAdapterException("IO Error while retrieving data from rrd db", e);
}
}
@Override
public String getEncoding() {
return "UTF-8";
}
@Override
public ZoneId getTimeZoneId() {
return ZoneId.systemDefault();
}
@Override
public String getSourceName() {
return "[RRD] " + rrdPaths.get(0).getFileName() + (rrdPaths.size() > 1 ? " + " + (rrdPaths.size() - 1) +
" more RRD file(s)" : "");
}
@Override
public Map<String, String> getParams() {
Map<String, String> params = new HashMap<>();
int i = 0;
for (Path rrdPath : rrdPaths) {
params.put("rrdPaths_" + i++, rrdPath.toString());
}
return params;
}
@Override
public void loadParams(Map<String, String> params) throws DataAdapterException {
this.rrdPaths = params.entrySet().stream()
.filter(entry -> entry.getKey().startsWith("rrdPaths_"))
.map(e -> Paths.get(e.getValue()))
.collect(Collectors.toList());
}
@Override
public void close() {
closeRrdDb();
cleanTempFiles();
super.close();
}
private RrdDb openRrdDb(Path rrdPath) throws IOException {
var factory = RrdBackendFactory.getFactory(prefs.rrd4jBackend.get().toString());
logger.debug(()-> "Opening rrd file using backend factory= " + factory.getName());
if ("text/xml".equalsIgnoreCase(Files.probeContentType(rrdPath))) {
logger.debug(() -> "Attempting to import as an rrd XML dump");
Path temp = Files.createTempFile(UserPreferences.getInstance().temporaryFilesRoot.get(), "binjr_", "_imported.rrd");
tempPathToCollect.add(temp);
return RrdDb.getBuilder()
.setBackendFactory(factory)
.setPath(temp.toUri())
.setReadOnly(true)
.setExternalPath(RrdDb.PREFIX_XML + rrdPath.toString())
.build();
}
try {
return RrdDb.getBuilder()
.setBackendFactory(factory)
.setPath(rrdPath.toUri())
.setReadOnly(true)
.build();
} catch (InvalidRrdException e) {
// Possibly a rrd db created with RrdTool.
// Try to convert and import.
logger.debug(() -> "Failed to open " + rrdPath + " as an Rrd4j db: attempting to import as an rrdTool db");
Path temp = Files.createTempFile(UserPreferences.getInstance().temporaryFilesRoot.get(), "binjr_", "_imported.rrd");
tempPathToCollect.add(temp);
return RrdDb.getBuilder()
.setBackendFactory(factory)
.setPath(temp.toUri())
.setReadOnly(true)
.setExternalPath(RrdDb.PREFIX_RRDTool + rrdPath.toString())
.build();
}
}
private void closeRrdDb() {
rrdDbMap.forEach((s, rrdDb) -> {
logger.debug(() -> "Closing RRD db " + s);
try {
rrdDb.close();
} catch (IOException e) {
logger.error("Error attempting to close RRD db " + s, e);
}
});
rrdDbMap.clear();
}
private void cleanTempFiles() {
//Cleaning up temp files used to import rrdtool files
tempPathToCollect.forEach(p -> {
logger.debug(() -> "Deleting temp file " + p);
try {
Files.delete(p);
} catch (IOException e) {
logger.error("Failed to delete temp file", e);
}
});
tempPathToCollect.clear();
}
}
binjr/binjr-adapter-rrd4j/src/main/java/eu/binjr/sources/rrd4j/adapters/Rrd4jFileAdapterDialog.java
+154
View File
@@ -0,0 +1,154 @@
/*
* Copyright 2018-2023 Frederic Thevenet
*
* Licensed under the Apache License, Version 2.0 (the "License");
* you may not use this file except in compliance with the License.
* You may obtain a copy of the License at
*
* http://www.apache.org/licenses/LICENSE-2.0
*
* Unless required by applicable law or agreed to in writing, software
* distributed under the License is distributed on an "AS IS" BASIS,
* WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
* See the License for the specific language governing permissions and
* limitations under the License.
*/
package eu.binjr.sources.rrd4j.adapters;
import eu.binjr.common.javafx.controls.NodeUtils;
import eu.binjr.common.logging.Logger;
import eu.binjr.common.preferences.MostRecentlyUsedList;
import eu.binjr.core.data.adapters.DataAdapter;
import eu.binjr.core.data.exceptions.CannotInitializeDataAdapterException;
import eu.binjr.core.data.exceptions.DataAdapterException;
import eu.binjr.core.dialogs.Dialogs;
import eu.binjr.core.preferences.AppEnvironment;
import eu.binjr.core.preferences.UserHistory;
import javafx.application.Platform;
import javafx.event.ActionEvent;
import javafx.geometry.Pos;
import javafx.scene.Node;
import javafx.scene.control.*;
import javafx.scene.layout.HBox;
import javafx.scene.layout.Region;
import javafx.stage.FileChooser;
import java.io.File;
import java.nio.file.Path;
import java.nio.file.Paths;
import java.util.Arrays;
import java.util.Collection;
import java.util.List;
import java.util.stream.Collectors;
/**
* A dialog box that returns a {@link Rrd4jFileAdapterDialog} built according to user inputs.
*
* @author Frederic Thevenet
*/
public class Rrd4jFileAdapterDialog extends Dialog<Collection<DataAdapter>> {
private static final Logger logger = Logger.create(Rrd4jFileAdapterDialog.class);
private static final String BINJR_SOURCES = "binjr/sources";
private Collection<DataAdapter> result = null;
private final TextField pathsField;
private final MostRecentlyUsedList<Path> mostRecentRrdFiles =
UserHistory.getInstance().pathMostRecentlyUsedList("mostRecentRrdFiles", 20, false);
/**
* Initializes a new instance of the {@link Rrd4jFileAdapterDialog} class.
*
* @param owner the owner window for the dialog
*/
public Rrd4jFileAdapterDialog(Node owner) {
if (owner != null) {
this.initOwner(NodeUtils.getStage(owner));
}
this.setTitle("Source");
Button browseButton = new Button("Browse");
pathsField = new TextField();
HBox pathHBox = new HBox();
pathHBox.setSpacing(10);
pathHBox.setAlignment(Pos.CENTER);
pathHBox.getChildren().addAll(pathsField, browseButton);
browseButton.setPrefWidth(-1);
pathsField.setPrefWidth(400);
DialogPane dialogPane = new DialogPane();
dialogPane.setHeaderText("Add RRD file(s)");
dialogPane.getButtonTypes().addAll(ButtonType.OK, ButtonType.CANCEL);
dialogPane.setGraphic(new Region());
dialogPane.getGraphic().getStyleClass().addAll("source-icon", "dialog-icon");
dialogPane.setContent(pathHBox);
this.setDialogPane(dialogPane);
browseButton.setOnAction(event -> {
File selectedFile = displayFileChooser((Node) event.getSource());
if (selectedFile != null) {
pathsField.setText(selectedFile.getPath());
}
});
Button okButton = (Button) dialogPane.lookupButton(ButtonType.OK);
Platform.runLater(pathsField::requestFocus);
okButton.addEventFilter(ActionEvent.ACTION, ae -> {
try {
result = List.of(getDataAdapter());
} catch (CannotInitializeDataAdapterException e) {
Dialogs.notifyError("Error initializing adapter to source", e, Pos.CENTER, pathsField);
ae.consume();
} catch (DataAdapterException e) {
Dialogs.notifyError("Error with the adapter to source", e, Pos.CENTER, pathsField);
ae.consume();
} catch (Throwable e) {
Dialogs.notifyError("Unexpected error while retrieving data adapter", e, Pos.CENTER, pathsField);
ae.consume();
}
});
this.setResultConverter(dialogButton -> {
ButtonBar.ButtonData data = dialogButton == null ? null : dialogButton.getButtonData();
if (data == ButtonBar.ButtonData.OK_DONE) {
return result;
}
return null;
}
);
// Workaround JDK-8179073 (ref: https://bugs.openjdk.java.net/browse/JDK-8179073)
this.setResizable(AppEnvironment.getInstance().isResizableDialogs());
}
private File displayFileChooser(Node owner) {
try {
FileChooser fileChooser = new FileChooser();
fileChooser.setTitle("Open Rrd4j File(s)");
fileChooser.getExtensionFilters().add(new FileChooser.ExtensionFilter("RRD binary files", "*.rrd"));
fileChooser.getExtensionFilters().add(new FileChooser.ExtensionFilter("RRD XML dumps", "*.xml"));
fileChooser.getExtensionFilters().add(new FileChooser.ExtensionFilter("All files", "*.*", "*"));
Dialogs.getInitialDir(mostRecentRrdFiles).ifPresent(fileChooser::setInitialDirectory);
List<File> rrdFiles = fileChooser.showOpenMultipleDialog(NodeUtils.getStage(owner));
if (rrdFiles != null) {
pathsField.setText(rrdFiles.stream().map(File::getPath).collect(Collectors.joining(";")));
}
} catch (Exception e) {
Dialogs.notifyException("Error while displaying file chooser: " + e.getMessage(), e, owner);
}
return null;
}
/**
* Returns an instance of {@link Rrd4jFileAdapter}
*
* @return an instance of {@link Rrd4jFileAdapter}
* @throws DataAdapterException if the provided parameters are invalid
*/
private DataAdapter<Double> getDataAdapter() throws DataAdapterException {
if (pathsField.getText().isBlank()){
throw new CannotInitializeDataAdapterException("Path cannot be blank");
}
List<Path> rrdFiles = Arrays.stream(pathsField.getText().split(";")).map(Paths::get).collect(Collectors.toList());
rrdFiles.forEach(mostRecentRrdFiles::push);
return new Rrd4jFileAdapter(rrdFiles);
}
}
binjr/binjr-adapter-rrd4j/src/main/java/eu/binjr/sources/rrd4j/adapters/Rrd4jFileAdapterPreferences.java
+46
View File
@@ -0,0 +1,46 @@
/*
* Copyright 2020 Frederic Thevenet
*
* Licensed under the Apache License, Version 2.0 (the "License");
* you may not use this file except in compliance with the License.
* You may obtain a copy of the License at
*
* http://www.apache.org/licenses/LICENSE-2.0
*
* Unless required by applicable law or agreed to in writing, software
* distributed under the License is distributed on an "AS IS" BASIS,
* WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
* See the License for the specific language governing permissions and
* limitations under the License.
*/
package eu.binjr.sources.rrd4j.adapters;
import eu.binjr.common.preferences.ObservablePreference;
import eu.binjr.core.data.adapters.DataAdapter;
import eu.binjr.core.data.adapters.DataAdapterPreferences;
/**
* Defines the preferences associated with the RRD4J adapter.
*
* @author Frederic Thevenet
*/
public class Rrd4jFileAdapterPreferences extends DataAdapterPreferences {
/**
* The backend to be used by RRD4J to mount rrd files.
*/
public final ObservablePreference<Rrd4jBackendType> rrd4jBackend =
enumPreference(Rrd4jBackendType.class, "rrd4jBackend", Rrd4jBackendType.NIO);
/**
* Initialize a new instance of the {@link Rrd4jFileAdapterPreferences} class associated to
* a {@link DataAdapter} instance.
*
* @param dataAdapterClass the associated {@link DataAdapter}
*/
public Rrd4jFileAdapterPreferences(Class<? extends DataAdapter<?>> dataAdapterClass) {
super(dataAdapterClass);
}
}
binjr/binjr-adapter-rrd4j/src/main/java/eu/binjr/sources/rrd4j/adapters/Rrd4jFileDataAdapterInfo.java
+54
View File
@@ -0,0 +1,54 @@
/*
* Copyright 2018-2023 Frederic Thevenet
*
* Licensed under the Apache License, Version 2.0 (the "License");
* you may not use this file except in compliance with the License.
* You may obtain a copy of the License at
*
* http://www.apache.org/licenses/LICENSE-2.0
*
* Unless required by applicable law or agreed to in writing, software
* distributed under the License is distributed on an "AS IS" BASIS,
* WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
* See the License for the specific language governing permissions and
* limitations under the License.
*/
package eu.binjr.sources.rrd4j.adapters;
import eu.binjr.core.data.adapters.AdapterMetadata;
import eu.binjr.core.data.adapters.BaseDataAdapterInfo;
import eu.binjr.core.data.adapters.SourceLocality;
import eu.binjr.core.data.adapters.VisualizationType;
import eu.binjr.core.data.exceptions.CannotInitializeDataAdapterException;
import eu.binjr.core.preferences.AppEnvironment;
/**
* Defines the metadata associated with the Rrd4jFileDataAdapter
*
* @author Frederic Thevenet
*/
@AdapterMetadata(
name = "RRD",
description = "RRD Data Adapter",
copyright = AppEnvironment.COPYRIGHT_NOTICE,
license = AppEnvironment.LICENSE,
siteUrl = AppEnvironment.HTTP_WWW_BINJR_EU,
adapterClass = Rrd4jFileAdapter.class,
dialogClass = Rrd4jFileAdapterDialog.class,
preferencesClass = Rrd4jFileAdapterPreferences.class,
sourceLocality = SourceLocality.LOCAL,
apiLevel = AppEnvironment.PLUGIN_API_LEVEL,
visualizationType = VisualizationType.CHARTS
)
public class Rrd4jFileDataAdapterInfo extends BaseDataAdapterInfo {
/**
* Initialises a new instance of the {@link Rrd4jFileDataAdapterInfo} class.
*
* @throws CannotInitializeDataAdapterException if an error occurs initializing the adapter.
*/
public Rrd4jFileDataAdapterInfo() throws CannotInitializeDataAdapterException {
super(Rrd4jFileDataAdapterInfo.class);
}
}
binjr/binjr-adapter-rrd4j/src/main/resources/META-INF/services/eu.binjr.core.data.adapters.DataAdapterInfo
+18
View File
@@ -0,0 +1,18 @@
#
# Copyright 2018 Frederic Thevenet
#
# Licensed under the Apache License, Version 2.0 (the "License");
# you may not use this file except in compliance with the License.
# You may obtain a copy of the License at
#
# http://www.apache.org/licenses/LICENSE-2.0
#
# Unless required by applicable law or agreed to in writing, software
# distributed under the License is distributed on an "AS IS" BASIS,
# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
# See the License for the specific language governing permissions and
# limitations under the License.
#
# RRD file Data adapter service implementation
eu.binjr.sources.rrd4j.adapters.Rrd4jFileDataAdapterInfo
binjr/binjr-adapter-text/build.gradle
+31
View File
@@ -0,0 +1,31 @@
/*
* Copyright 2020-2021 Frederic Thevenet
*
* Licensed under the Apache License, Version 2.0 (the "License");
* you may not use this file except in compliance with the License.
* You may obtain a copy of the License at
*
* http://www.apache.org/licenses/LICENSE-2.0
*
* Unless required by applicable law or agreed to in writing, software
* distributed under the License is distributed on an "AS IS" BASIS,
* WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
* See the License for the specific language governing permissions and
* limitations under the License.
*/
dependencies {
compileOnly project(':binjr-core')
}
jar {
manifest {
attributes(
'Specification-Title': project.name,
'Specification-Version': project.version,
'Implementation-Title': project.name,
'Implementation-Version': project.version,
'Build-Number': BINJR_BUILD_NUMBER
)
}
}
binjr/binjr-adapter-text/src/main/java/eu/binjr/sources/text/adapters/TextAdapterPreferences.java
+63
View File
@@ -0,0 +1,63 @@
/*
* Copyright 2020 Frederic Thevenet
*
* Licensed under the Apache License, Version 2.0 (the "License");
* you may not use this file except in compliance with the License.
* You may obtain a copy of the License at
*
* http://www.apache.org/licenses/LICENSE-2.0
*
* Unless required by applicable law or agreed to in writing, software
* distributed under the License is distributed on an "AS IS" BASIS,
* WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
* See the License for the specific language governing permissions and
* limitations under the License.
*/
package eu.binjr.sources.text.adapters;
import com.google.gson.Gson;
import eu.binjr.common.preferences.ObservablePreference;
import eu.binjr.core.data.adapters.DataAdapter;
import eu.binjr.core.data.adapters.DataAdapterPreferences;
/**
* Defines the preferences associated with the Text files adapter.
*/
public class TextAdapterPreferences extends DataAdapterPreferences {
private static final Gson gson = new Gson();
/**
* The default text panel font size preference.
*/
public ObservablePreference<Number> defaultTextViewFontSize = integerPreference("defaultTextViewFontSize", 10);
/**
* The filters used when scanning folders in the source filesystem.
*/
public ObservablePreference<String[]> folderFilters = objectPreference(String[].class,
"folderFilters",
new String[]{"*"},
gson::toJson,
s -> gson.fromJson(s, String[].class));
/**
* The filters used to prune file extensions to scan in the source filesystem.
*/
public ObservablePreference<String[]> fileExtensionFilters = objectPreference(String[].class,
"fileExtensionFilters",
new String[]{".xml", ".txt", ".env", ".properties", ".csv", ".log", "md", ".json", ".yml"},
gson::toJson,
s -> gson.fromJson(s, String[].class));
/**
* Initialize a new instance of the {@link TextAdapterPreferences} class associated to
* a {@link DataAdapter} instance.
*
* @param dataAdapterClass the associated {@link DataAdapter}
*/
public TextAdapterPreferences(Class<? extends DataAdapter<?>> dataAdapterClass) {
super(dataAdapterClass, false);
}
}
binjr/binjr-adapter-text/src/main/java/eu/binjr/sources/text/adapters/TextDataAdapter.java
+253
View File
@@ -0,0 +1,253 @@
/*
* Copyright 2020-2023 Frederic Thevenet
*
* Licensed under the Apache License, Version 2.0 (the "License");
* you may not use this file except in compliance with the License.
* You may obtain a copy of the License at
*
* http://www.apache.org/licenses/LICENSE-2.0
*
* Unless required by applicable law or agreed to in writing, software
* distributed under the License is distributed on an "AS IS" BASIS,
* WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
* See the License for the specific language governing permissions and
* limitations under the License.
*/
package eu.binjr.sources.text.adapters;
import com.google.gson.Gson;
import eu.binjr.common.io.FileSystemBrowser;
import eu.binjr.common.javafx.controls.TreeViewUtils;
import eu.binjr.common.logging.Logger;
import eu.binjr.common.logging.Profiler;
import eu.binjr.common.text.BinaryPrefixFormatter;
import eu.binjr.core.data.adapters.BaseDataAdapter;
import eu.binjr.core.data.adapters.SourceBinding;
import eu.binjr.core.data.adapters.TextFilesBinding;
import eu.binjr.core.data.exceptions.DataAdapterException;
import eu.binjr.core.data.timeseries.TextProcessor;
import eu.binjr.core.data.timeseries.TimeSeriesProcessor;
import eu.binjr.core.data.workspace.TimeSeriesInfo;
import eu.binjr.core.dialogs.Dialogs;
import javafx.scene.chart.XYChart;
import org.eclipse.fx.ui.controls.tree.FilterableTreeItem;
import java.io.BufferedReader;
import java.io.IOException;
import java.io.InputStreamReader;
import java.nio.charset.StandardCharsets;
import java.nio.file.Path;
import java.nio.file.Paths;
import java.time.Instant;
import java.time.ZoneId;
import java.time.ZonedDateTime;
import java.util.Arrays;
import java.util.HashMap;
import java.util.List;
import java.util.Map;
import java.util.stream.Collectors;
/**
* A {@link eu.binjr.core.data.adapters.DataAdapter} implementation to retrieve data from a text file.
*
* @author Frederic Thevenet
*/
public class TextDataAdapter extends BaseDataAdapter<String> {
private static final Logger logger = Logger.create(TextDataAdapter.class);
private static final Gson gson = new Gson();
private final BinaryPrefixFormatter binaryPrefixFormatter = new BinaryPrefixFormatter("###,###.## ");
private Path rootPath;
private FileSystemBrowser fileBrowser;
private String[] folderFilters;
private String[] fileExtensionsFilters;
/**
* Initializes a new instance of the {@link TextDataAdapter} class.
*
* @throws DataAdapterException if an error occurs while initializing the adapter.
*/
public TextDataAdapter() throws DataAdapterException {
super();
}
/**
* Initializes a new instance of the {@link TextDataAdapter} class from the provided {@link Path}
*
* @param rootPath the {@link Path} from which to load content.
* @param folderFilters a list of names of folders to inspect for content.
* @param fileExtensionsFilters a list of file extensions to inspect for content.
* @throws DataAdapterException if an error occurs initializing the adapter.
*/
public TextDataAdapter(Path rootPath, String[] folderFilters, String[] fileExtensionsFilters) throws DataAdapterException {
super();
this.rootPath = rootPath;
Map<String, String> params = new HashMap<>();
initParams(rootPath, folderFilters, fileExtensionsFilters);
}
@Override
public Map<String, String> getParams() {
Map<String, String> params = new HashMap<>();
params.put("rootPath", rootPath.toString());
params.put("folderFilters", gson.toJson(folderFilters));
params.put("fileExtensionsFilters", gson.toJson(fileExtensionsFilters));
return params;
}
@Override
public void loadParams(Map<String, String> params) throws DataAdapterException {
if (logger.isDebugEnabled()) {
logger.debug(() -> "TextDataAdapter params:");
params.forEach((s, s2) -> logger.debug(() -> "key=" + s + ", value=" + s2));
}
initParams(Paths.get(validateParameterNullity(params, "rootPath")),
gson.fromJson(validateParameterNullity(params, "folderFilters"), String[].class),
gson.fromJson(validateParameterNullity(params, "fileExtensionsFilters"), String[].class));
}
private void initParams(Path rootPath, String[] folderFilters, String[] fileExtensionsFilters) throws DataAdapterException {
this.rootPath = rootPath;
this.folderFilters = folderFilters;
this.fileExtensionsFilters = fileExtensionsFilters;
try {
this.fileBrowser = FileSystemBrowser.of(rootPath);
} catch (IOException e) {
throw new DataAdapterException("Could not create file system browser instance", e);
}
}
@Override
public FilterableTreeItem<SourceBinding> getBindingTree() throws DataAdapterException {
FilterableTreeItem<SourceBinding> rootNode = new FilterableTreeItem<>(
new TextFilesBinding.Builder()
.withLabel(getSourceName())
.withAdapter(this)
.build());
attachTextFilesTree(rootNode);
return rootNode;
}
private void attachTextFilesTree(FilterableTreeItem<SourceBinding> rootNode) throws DataAdapterException {
try (var p = Profiler.start("Building text binding tree", logger::perf)) {
Map<Path, FilterableTreeItem<SourceBinding>> nodeDict = new HashMap<>();
nodeDict.put(fileBrowser.toInternalPath("/"), rootNode);
for (var fsEntry : fileBrowser.listEntries(folderFilters, fileExtensionsFilters)) {
String fileName = fsEntry.getPath().getFileName().toString();
var attachTo = rootNode;
if (fsEntry.getPath().getParent() != null) {
attachTo = nodeDict.get(fsEntry.getPath().getParent());
if (attachTo == null) {
attachTo = makeBranchNode(nodeDict, fsEntry.getPath().getParent(), rootNode);
}
}
FilterableTreeItem<SourceBinding> filenode = new FilterableTreeItem<>(
new TextFilesBinding.Builder()
.withLabel(fileName + " (" + binaryPrefixFormatter.format(fsEntry.getSize()) + "B)")
.withPath(fsEntry.getPath().toString())
.withParent(attachTo.getValue())
.withAdapter(this)
.build());
attachTo.getInternalChildren().add(filenode);
}
TreeViewUtils.sortFromBranch(rootNode);
} catch (Exception e) {
Dialogs.notifyException("Error while enumerating files: " + e.getMessage(), e);
}
}
private FilterableTreeItem<SourceBinding> makeBranchNode(Map<Path, FilterableTreeItem<SourceBinding>> nodeDict,
Path path,
FilterableTreeItem<SourceBinding> root) {
var parent = root;
var rootPath = path.isAbsolute() ? path.getRoot() : path.getName(0);
for (int i = 0; i < path.getNameCount(); i++) {
Path current = rootPath.resolve(path.getName(i));
FilterableTreeItem<SourceBinding> filenode = nodeDict.get(current);
if (filenode == null) {
filenode = new FilterableTreeItem<>(
new TextFilesBinding.Builder()
.withLabel(current.getFileName().toString())
.withPath(path.toString())
.withParent(parent.getValue())
.withAdapter(this)
.build());
nodeDict.put(current, filenode);
parent.getInternalChildren().add(filenode);
}
parent = filenode;
rootPath = current;
}
return parent;
}
@Override
public Map<TimeSeriesInfo<String>, TimeSeriesProcessor<String>> fetchData(String path,
Instant start,
Instant end,
List<TimeSeriesInfo<String>> seriesInfos,
boolean bypassCache) throws DataAdapterException {
Map<TimeSeriesInfo<String>, TimeSeriesProcessor<String>> data = new HashMap<>();
for (var info : seriesInfos) {
try {
var proc = new TextProcessor();
proc.setData(List.of(new XYChart.Data<>(ZonedDateTime.now(), readTextFile(info.getBinding().getPath()))));
data.put(info, proc);
} catch (IOException e) {
throw new DataAdapterException("Error fetching text from " + info.getBinding().getPath(), e);
}
}
return data;
}
@Override
public String getEncoding() {
return "utf-8";
}
@Override
public ZoneId getTimeZoneId() {
return ZoneId.systemDefault();
}
@Override
public String getSourceName() {
return "[Text] " + (rootPath != null ? rootPath.getFileName() : "???");
}
@Override
public void onStart() throws DataAdapterException {
super.onStart();
}
@Override
public void close() {
if (fileBrowser != null) {
try {
fileBrowser.close();
} catch (IOException e) {
logger.error("An error occurred while closing file system browser instance: " + e.getMessage());
logger.debug(e);
}
}
super.close();
}
/**
* Read the content of the text file located at the provided path
*
* @param path the path of the text file to read
* @return The content of the text file, as a String
* @throws IOException if an error occurs while reading the file.
*/
public String readTextFile(String path) throws IOException {
try (Profiler ignored = Profiler.start("Extracting text from file " + path, logger::perf)) {
try (var reader = new BufferedReader(new InputStreamReader(fileBrowser.getData(path), StandardCharsets.UTF_8))) {
return reader.lines().collect(Collectors.joining("\n"));
}
}
}
}
binjr/binjr-adapter-text/src/main/java/eu/binjr/sources/text/adapters/TextDataAdapterDialog.java
+140
View File
@@ -0,0 +1,140 @@
/*
* Copyright 2020-2023 Frederic Thevenet
*
* Licensed under the Apache License, Version 2.0 (the "License");
* you may not use this file except in compliance with the License.
* You may obtain a copy of the License at
*
* http://www.apache.org/licenses/LICENSE-2.0
*
* Unless required by applicable law or agreed to in writing, software
* distributed under the License is distributed on an "AS IS" BASIS,
* WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
* See the License for the specific language governing permissions and
* limitations under the License.
*/
package eu.binjr.sources.text.adapters;
import com.google.gson.Gson;
import eu.binjr.common.javafx.controls.NodeUtils;
import eu.binjr.common.logging.Logger;
import eu.binjr.core.data.adapters.DataAdapter;
import eu.binjr.core.data.adapters.DataAdapterFactory;
import eu.binjr.core.data.exceptions.CannotInitializeDataAdapterException;
import eu.binjr.core.data.exceptions.DataAdapterException;
import eu.binjr.core.data.exceptions.NoAdapterFoundException;
import eu.binjr.core.dialogs.DataAdapterDialog;
import eu.binjr.core.dialogs.Dialogs;
import javafx.geometry.HPos;
import javafx.geometry.Insets;
import javafx.geometry.Side;
import javafx.geometry.VPos;
import javafx.scene.Node;
import javafx.scene.control.ContextMenu;
import javafx.scene.control.Label;
import javafx.scene.control.MenuItem;
import javafx.scene.control.TextField;
import javafx.scene.layout.GridPane;
import javafx.scene.layout.Priority;
import javafx.stage.DirectoryChooser;
import javafx.stage.FileChooser;
import java.io.File;
import java.nio.file.Files;
import java.nio.file.Path;
import java.nio.file.Paths;
import java.util.Collection;
import java.util.List;
/**
* An implementation of the {@link DataAdapterDialog} class that presents a dialog box to retrieve the parameters specific {@link TextDataAdapterDialog}
*
* @author Frederic Thevenet
*/
public class TextDataAdapterDialog extends DataAdapterDialog<Path> {
private static final Logger logger = Logger.create(TextDataAdapterDialog.class);
private final TextField extensionFiltersTextField;
private final TextAdapterPreferences prefs;
private static final Gson gson = new Gson();
/**
* Initializes a new instance of the {@link TextDataAdapterDialog} class.
*
* @param owner the owner window for the dialog
* @throws NoAdapterFoundException if no adapter could be found to get preferences for.
*/
public TextDataAdapterDialog(Node owner) throws NoAdapterFoundException {
super(owner, Mode.PATH, "mostRecentTextArchives", false);
this.prefs = (TextAdapterPreferences) DataAdapterFactory.getInstance().getAdapterPreferences(TextDataAdapter.class.getName());
setDialogHeaderText("Add a TExt File, Zip Archive or Folder");
extensionFiltersTextField = new TextField(gson.toJson(prefs.fileExtensionFilters.get()));
var label = new Label("Extensions:");
GridPane.setConstraints(label, 0, 1, 1, 1, HPos.LEFT, VPos.CENTER, Priority.ALWAYS, Priority.ALWAYS, new Insets(4, 0, 4, 0));
GridPane.setConstraints(extensionFiltersTextField, 1, 1, 1, 1, HPos.LEFT, VPos.CENTER, Priority.ALWAYS, Priority.ALWAYS, new Insets(4, 0, 4, 0));
getParamsGridPane().getChildren().addAll(label, extensionFiltersTextField);
}
@Override
protected File displayFileChooser(Node owner) {
try {
ContextMenu sourceMenu = new ContextMenu();
MenuItem fileMenuItem = new MenuItem("Text file");
fileMenuItem.setOnAction(eventHandler -> {
FileChooser fileChooser = new FileChooser();
fileChooser.setTitle("Open Text File");
fileChooser.getExtensionFilters().add(new FileChooser.ExtensionFilter("Text files", "*.txt", "*.text"));
fileChooser.getExtensionFilters().add(new FileChooser.ExtensionFilter("All files", "*.*", "*"));
Dialogs.getInitialDir(getMostRecentList()).ifPresent(fileChooser::setInitialDirectory);
File selectedFile = fileChooser.showOpenDialog(NodeUtils.getStage(owner));
if (selectedFile != null) {
setSourceUri(selectedFile.getPath());
}
});
sourceMenu.getItems().add(fileMenuItem);
MenuItem menuItem = new MenuItem("Zip file");
menuItem.setOnAction(eventHandler -> {
FileChooser fileChooser = new FileChooser();
fileChooser.setTitle("Open Zip Archive");
fileChooser.getExtensionFilters().add(new FileChooser.ExtensionFilter("Zip archive", "*.zip"));
fileChooser.getExtensionFilters().add(new FileChooser.ExtensionFilter("All files", "*.*", "*"));
Dialogs.getInitialDir(getMostRecentList()).ifPresent(fileChooser::setInitialDirectory);
File selectedFile = fileChooser.showOpenDialog(NodeUtils.getStage(owner));
if (selectedFile != null) {
setSourceUri(selectedFile.getPath());
}
});
sourceMenu.getItems().add(menuItem);
MenuItem folderMenuItem = new MenuItem("Folder");
folderMenuItem.setOnAction(eventHandler -> {
DirectoryChooser dirChooser = new DirectoryChooser();
dirChooser.setTitle("Open Folder");
Dialogs.getInitialDir(getMostRecentList()).ifPresent(dirChooser::setInitialDirectory);
File selectedFile = dirChooser.showDialog(NodeUtils.getStage(owner));
if (selectedFile != null) {
setSourceUri(selectedFile.getPath());
}
});
sourceMenu.getItems().add(folderMenuItem);
sourceMenu.show(owner, Side.RIGHT, 0, 0);
} catch (Exception e) {
Dialogs.notifyException("Error while displaying file chooser: " + e.getMessage(), e, owner);
}
return null;
}
@Override
protected Collection<DataAdapter> getDataAdapters() throws DataAdapterException {
Path path = Paths.get(getSourceUri());
if (!Files.exists(path)) {
throw new CannotInitializeDataAdapterException("Cannot find " + getSourceUri());
}
if (!path.isAbsolute()) {
throw new CannotInitializeDataAdapterException("The provided path is not valid.");
}
getMostRecentList().push(path);
prefs.fileExtensionFilters.set(gson.fromJson(extensionFiltersTextField.getText(), String[].class));
return List.of(new TextDataAdapter(path, prefs.folderFilters.get(), prefs.fileExtensionFilters.get()));
}
}
binjr/binjr-adapter-text/src/main/java/eu/binjr/sources/text/adapters/TextDataAdapterInfo.java
+56
View File
@@ -0,0 +1,56 @@
/*
* Copyright 2020-2023 Frederic Thevenet
*
* Licensed under the Apache License, Version 2.0 (the "License");
* you may not use this file except in compliance with the License.
* You may obtain a copy of the License at
*
* http://www.apache.org/licenses/LICENSE-2.0
*
* Unless required by applicable law or agreed to in writing, software
* distributed under the License is distributed on an "AS IS" BASIS,
* WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
* See the License for the specific language governing permissions and
* limitations under the License.
*/
package eu.binjr.sources.text.adapters;
import eu.binjr.core.data.adapters.AdapterMetadata;
import eu.binjr.core.data.adapters.BaseDataAdapterInfo;
import eu.binjr.core.data.adapters.SourceLocality;
import eu.binjr.core.data.adapters.VisualizationType;
import eu.binjr.core.data.exceptions.CannotInitializeDataAdapterException;
import eu.binjr.core.preferences.AppEnvironment;
/**
* Defines the metadata associated with TextDataAdapterInfo.
*
* @author Frederic Thevenet
*/
@AdapterMetadata(
name = "Text",
description = "Text Files Data Adapter",
copyright = AppEnvironment.COPYRIGHT_NOTICE,
license = AppEnvironment.LICENSE,
siteUrl = AppEnvironment.HTTP_WWW_BINJR_EU,
adapterClass = TextDataAdapter.class,
dialogClass = TextDataAdapterDialog.class,
preferencesClass = TextAdapterPreferences.class,
sourceLocality = SourceLocality.LOCAL,
apiLevel = AppEnvironment.PLUGIN_API_LEVEL,
visualizationType = VisualizationType.TEXT
)
public class TextDataAdapterInfo extends BaseDataAdapterInfo {
/**
* Initialises a new instance of the {@link TextDataAdapterInfo} class.
*
* @throws CannotInitializeDataAdapterException if the adapter's initialization failed
*/
public TextDataAdapterInfo() throws CannotInitializeDataAdapterException {
super(TextDataAdapterInfo.class);
}
}
binjr/binjr-adapter-text/src/main/resources/META-INF/services/eu.binjr.core.data.adapters.DataAdapterInfo
+18
View File
@@ -0,0 +1,18 @@
#
# Copyright 2020 Frederic Thevenet
#
# Licensed under the Apache License, Version 2.0 (the "License");
# you may not use this file except in compliance with the License.
# You may obtain a copy of the License at
#
# http://www.apache.org/licenses/LICENSE-2.0
#
# Unless required by applicable law or agreed to in writing, software
# distributed under the License is distributed on an "AS IS" BASIS,
# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
# See the License for the specific language governing permissions and
# limitations under the License.
#
# Text Data adapter service implementation
# eu.binjr.sources.text.adapters.TextDataAdapterInfo
binjr/binjr-app/README.md
+3
View File
@@ -0,0 +1,3 @@
# binjr-app
This module contains no actual code; it simply ties all other modules together (by depending on all of them) in order to make it easy to to start the complete application (core + all plugins) from Gradle.
binjr/binjr-app/build.gradle
+38
View File
@@ -0,0 +1,38 @@
/*
* Copyright 2018-2023 Frederic Thevenet
*
* Licensed under the Apache License, Version 2.0 (the "License");
* you may not use this file except in compliance with the License.
* You may obtain a copy of the License at
*
* http://www.apache.org/licenses/LICENSE-2.0
*
* Unless required by applicable law or agreed to in writing, software
* distributed under the License is distributed on an "AS IS" BASIS,
* WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
* See the License for the specific language governing permissions and
* limitations under the License.
*/
dependencies {
implementation project(':binjr-core')
implementation project(':binjr-adapter-csv')
implementation project(':binjr-adapter-jfr')
implementation project(':binjr-adapter-jrds')
implementation project(':binjr-adapter-logs')
implementation project(':binjr-adapter-netdata')
implementation project(':binjr-adapter-rrd4j')
implementation project(':binjr-adapter-text')
runtimeOnly "org.openjfx:javafx-base:$OPENJFX_VERSION:$OPENJFX_PLATEFORM_CLASSIFIER"
runtimeOnly "org.openjfx:javafx-graphics:$OPENJFX_VERSION:$OPENJFX_PLATEFORM_CLASSIFIER"
runtimeOnly "org.openjfx:javafx-controls:$OPENJFX_VERSION:$OPENJFX_PLATEFORM_CLASSIFIER"
runtimeOnly "org.openjfx:javafx-fxml:$OPENJFX_VERSION:$OPENJFX_PLATEFORM_CLASSIFIER"
runtimeOnly "org.openjfx:javafx-swing:$OPENJFX_VERSION:$OPENJFX_PLATEFORM_CLASSIFIER"
}
mainClassName = 'eu.binjr.core.Bootstrap'
run {
systemProperties System.getProperties()
}
binjr/binjr-core/README.md
+14
View File
@@ -0,0 +1,14 @@
# binjr-core
[![Maven Central](https://img.shields.io/maven-central/v/eu.binjr/binjr-core.svg?label=Maven%20Central&style=flat-square)](https://search.maven.org/search?q=g:%22eu.binjr%22%20AND%20a:%22binjr-core%22)
This is the main module for the application. It is responsible for :
* The presentation layer and interaction with the end-user.
* The management and persistence of user sessions:
- A single session is organised as a [Workspace](https://github.com/binjr/binjr/tree/master/binjr-core/src/main/java/eu/binjr/core/data/workspace/Workspace.java).
- A workspace can hold an arbitrary number of [Worksheets](https://github.com/binjr/binjr/tree/master/binjr-core/src/main/java/eu/binjr/core/data/workspace/Worksheet.java) and [Sources](https://github.com/binjr/binjr/tree/master/binjr-core/src/main/java/eu/binjr/core/data/workspace/Source.java)
- A worksheet displays data from time-series from any sources on one or more graphs, all sharing a single timeline.
- A source is backed by a [DataAdapter](https://github.com/binjr/binjr/tree/master/binjr-core/src/main/java/eu/binjr/core/data/adapters/DataAdapter.java), a component which handles the actual communication to a data source.
- A workspace can be saved (i.e. serialized to XML and written to a file) by the end-user at any time to be restored later on.
* Loading and instantiating DataAdapters for the various supported data sources, packaged in other modules.
* Exposing the [DataAdapter API](https://github.com/binjr/binjr/tree/master/binjr-core/src/main/java/eu/binjr/core/data/adapters) to other modules
binjr/binjr-core/build.gradle
+78
View File
@@ -0,0 +1,78 @@
/*
* Copyright 2018-2024 Frederic Thevenet
*
* Licensed under the Apache License, Version 2.0 (the "License");
* you may not use this file except in compliance with the License.
* You may obtain a copy of the License at
*
* http://www.apache.org/licenses/LICENSE-2.0
*
* Unless required by applicable law or agreed to in writing, software
* distributed under the License is distributed on an "AS IS" BASIS,
* WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
* See the License for the specific language governing permissions and
* limitations under the License.
*/
dependencies {
api 'org.apache.logging.log4j:log4j-core:2.22.0'
api 'org.apache.logging.log4j:log4j-jcl:2.22.0'
implementation 'org.apache.logging.log4j:log4j-slf4j-impl:2.22.0'
api 'org.controlsfx:controlsfx:11.2.1'
implementation 'org.gillius:jfxutils:1.0'
api 'com.google.code.gson:gson:2.10.1'
api 'org.apache.commons:commons-csv:1.10.0'
api 'jakarta.xml.bind:jakarta.xml.bind-api:4.0.2'
implementation 'org.glassfish.jaxb:jaxb-runtime:4.0.5'
implementation 'org.bouncycastle:bcpg-jdk18on:1.78.1'
implementation 'org.reflections:reflections:0.9.12'
api 'org.fxmisc.richtext:richtextfx:0.11.0'
api 'org.apache.lucene:lucene-core:9.10.0'
api 'org.apache.lucene:lucene-facet:9.10.0'
implementation 'org.apache.lucene:lucene-queryparser:9.10.0'
implementation 'org.apache.lucene:lucene-analysis-common:9.10.0'
implementation 'com.github.ben-manes.caffeine:caffeine:3.1.8'
// Only to prepare POM for Maven central upload
compileOnlyApi "org.openjfx:javafx-base:${OPENJFX_VERSION}"
compileOnlyApi "org.openjfx:javafx-graphics:${OPENJFX_VERSION}"
compileOnlyApi "org.openjfx:javafx-controls:${OPENJFX_VERSION}"
compileOnlyApi "org.openjfx:javafx-fxml:${OPENJFX_VERSION}"
compileOnlyApi "org.openjfx:javafx-swing:${OPENJFX_VERSION}"
}
compileJava {
doFirst {
options.compilerArgs = [
'--module-path', classpath.asPath,
'--add-modules', 'javafx.controls,javafx.fxml,javafx.swing'
]
}
}
jar {
manifest {
attributes(
'Main-Class': 'eu.binjr.core.Bootstrap',
'Specification-Title': project.name,
'Specification-Version': project.version,
'Implementation-Title': project.name,
'Implementation-Version': project.version,
'SplashScreen-Image': 'images / splashscreen.png',
'Build-Number': BINJR_BUILD_NUMBER
)
}
}
test {
useJUnitPlatform()
}
repositories {
mavenCentral()
}
binjr/binjr-core/src/main/java/eu/binjr/common/auth/CredentialsEntry.java
+103
View File
@@ -0,0 +1,103 @@
/*
* Copyright 2017-2018 Frederic Thevenet
*
* Licensed under the Apache License, Version 2.0 (the "License");
* you may not use this file except in compliance with the License.
* You may obtain a copy of the License at
*
* http://www.apache.org/licenses/LICENSE-2.0
*
* Unless required by applicable law or agreed to in writing, software
* distributed under the License is distributed on an "AS IS" BASIS,
* WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
* See the License for the specific language governing permissions and
* limitations under the License.
*/
package eu.binjr.common.auth;
/**
* Defines an immutable set of credentials
*
* @author Frederic Thevenet
*/
public class CredentialsEntry {
private final String login;
private final char[] pwd;
private final boolean filled;
/**
* An empty credential set.
* <p>This specific credential set should be used to intend that no credentials where explicitly provided</p>
*/
public static final CredentialsEntry EMPTY = new CredentialsEntry("", new char[0], false);
/**
* A canceled credential set.
* <p>This specific credential set should be used to intend that the query for explicit credentials was initiated but canceled</p>
*/
public static final CredentialsEntry CANCELLED = new CredentialsEntry("", new char[0], true);
/**
* Initializes a new instance of the {@link CredentialsEntry} class
*
* @param login the principle's login
* @param password the principle's password
*/
public CredentialsEntry(String login, char[] password) {
this(login, password, true);
}
/**
* Creates a deep clone of the provided {@link CredentialsEntry} instance.
*
* @param credentials the {@link CredentialsEntry} instance to copy.
* @return a deep clone of the provided {@link CredentialsEntry} instance.
*/
public static CredentialsEntry copyOf(CredentialsEntry credentials) {
return new CredentialsEntry(credentials.login, credentials.pwd, credentials.filled);
}
private CredentialsEntry(String login, char[] password, boolean filled) {
this.login = login;
this.pwd = password;
this.filled = filled;
}
/**
* Returns the principle's password
*
* @return the password
*/
public char[] getPwd() {
return pwd;
}
/**
* Returns the principle's login
*
* @return the principle's login
*/
public String getLogin() {
return login;
}
/**
* Returns true if the current set of credentials have been filled-in, false otherwise.
*
* @return true if the current set of credentials have been filled-in, false otherwise.
*/
public boolean isFilled() {
return filled;
}
/**
* Overwrites the array backing the principle's password property.
*/
public void clearPassword() {
for (int i = 0; i < pwd.length; i++) {
pwd[i] = '#';
}
}
}
binjr/binjr-core/src/main/java/eu/binjr/common/auth/JfxKrb5LoginModule.java
+116
View File
@@ -0,0 +1,116 @@
/*
* Copyright 2017-2021 Frederic Thevenet
*
* Licensed under the Apache License, Version 2.0 (the "License");
* you may not use this file except in compliance with the License.
* You may obtain a copy of the License at
*
* http://www.apache.org/licenses/LICENSE-2.0
*
* Unless required by applicable law or agreed to in writing, software
* distributed under the License is distributed on an "AS IS" BASIS,
* WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
* See the License for the specific language governing permissions and
* limitations under the License.
*/
package eu.binjr.common.auth;
import com.sun.security.auth.module.Krb5LoginModule;
import eu.binjr.common.logging.Logger;
import eu.binjr.core.dialogs.Dialogs;
import javafx.application.Platform;
import javafx.stage.StageStyle;
import javafx.util.Pair;
import org.controlsfx.dialog.LoginDialog;
import javax.security.auth.Subject;
import javax.security.auth.callback.*;
import java.util.Map;
import java.util.Optional;
import java.util.concurrent.CountDownLatch;
import java.util.concurrent.TimeUnit;
import java.util.concurrent.TimeoutException;
/**
* An extension of the Krb5LoginModule that displays a JavaFX dialog to obtain credentials from the end-user if need be.
*
* @author Frederic Thevenet
*/
public class JfxKrb5LoginModule extends Krb5LoginModule {
private static final Logger logger = Logger.create(JfxKrb5LoginModule.class);
private CredentialsEntry credentials = CredentialsEntry.EMPTY;
@Override
public void initialize(Subject subject, CallbackHandler callbackHandler, Map<String, ?> sharedState, Map<String, ?> options) {
super.initialize(subject, callbacks -> {
try {
credentials = obtainCredentials(CredentialsEntry.copyOf(credentials));
for (Callback callback : callbacks) {
if (callback instanceof NameCallback nc) {
nc.setName(credentials.getLogin());
} else if (callback instanceof PasswordCallback pc) {
pc.setPassword(credentials.getPwd());
credentials.clearPassword();
} else {
throw new UnsupportedCallbackException(callback, "Unknown Callback");
}
}
} catch (InterruptedException | TimeoutException e) {
logger.error("An exception occurred while retrieving credentials - " + e.getMessage());
if (logger.isDebugEnabled()) {
logger.debug("Stack trace", e);
}
}
}, sharedState, options);
}
private synchronized CredentialsEntry obtainCredentials(CredentialsEntry credentialsEntry) throws InterruptedException, TimeoutException {
if (credentialsEntry.isFilled()) {
return credentialsEntry;
}
AsyncResult<CredentialsEntry> future = new AsyncResult<>();
Platform.runLater(() -> {
LoginDialog dlg = new LoginDialog(null, null);
dlg.setHeaderText("Enter login credentials");
dlg.setTitle("Login");
dlg.initStyle(StageStyle.UTILITY);
Dialogs.setAlwaysOnTop(dlg);
Optional<Pair<String, String>> res = dlg.showAndWait();
if (res.isPresent()) {
CredentialsEntry newCreds = new CredentialsEntry(dlg.getResult().getKey(), dlg.getResult().getValue().toCharArray());
future.put(newCreds);
} else {
future.put(CredentialsEntry.CANCELLED);
}
});
return future.get();
}
private class AsyncResult<F> {
private final CountDownLatch latch = new CountDownLatch(1);
private F value;
public boolean isDone() {
return latch.getCount() == 0;
}
public F get() throws InterruptedException {
latch.await();
return value;
}
public F get(long timeout, TimeUnit unit) throws InterruptedException, TimeoutException {
if (latch.await(timeout, unit)) {
return value;
} else {
throw new TimeoutException();
}
}
private void put(F result) {
value = result;
latch.countDown();
}
}
}
binjr/binjr-core/src/main/java/eu/binjr/common/cache/Cacheable.java
Vendored
+39
View File
@@ -0,0 +1,39 @@
/*
* Copyright 2017-2018 Frederic Thevenet
*
* Licensed under the Apache License, Version 2.0 (the "License");
* you may not use this file except in compliance with the License.
* You may obtain a copy of the License at
*
* http://www.apache.org/licenses/LICENSE-2.0
*
* Unless required by applicable law or agreed to in writing, software
* distributed under the License is distributed on an "AS IS" BASIS,
* WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
* See the License for the specific language governing permissions and
* limitations under the License.
*/
package eu.binjr.common.cache;
/**
* Indicates that the implementing class can be cached using {@link LRUMapCapacityBound} or {@link LRUMapSizeBound}
*
* @param <T> The type for cached values.
*/
public interface Cacheable<T> {
/**
* Returns the cached value
*
* @return the cached value/
*/
T getValue();
/**
* Returns the size in bytes of the cached value.
*
* @return The size in bytes of the cached value.
*/
long getSize();
}
binjr/binjr-core/src/main/java/eu/binjr/common/cache/LRUMapCapacityBound.java
Vendored
+58
View File
@@ -0,0 +1,58 @@
/*
* Copyright 2017-2018 Frederic Thevenet
*
* Licensed under the Apache License, Version 2.0 (the "License");
* you may not use this file except in compliance with the License.
* You may obtain a copy of the License at
*
* http://www.apache.org/licenses/LICENSE-2.0
*
* Unless required by applicable law or agreed to in writing, software
* distributed under the License is distributed on an "AS IS" BASIS,
* WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
* See the License for the specific language governing permissions and
* limitations under the License.
*/
package eu.binjr.common.cache;
import java.util.LinkedHashMap;
import java.util.Map;
/**
* Finite capacity map with a Least Recent Used eviction policy
*
* @param <K> type of keys
* @param <V> type of values
* @author Frederic Thevenet
*/
@Deprecated
public class LRUMapCapacityBound<K, V> extends LinkedHashMap<K, V> {
private int cacheSize;
/**
* Initializes a new instance of the {@link LRUMapCapacityBound} class with the specified capacity
*
* @param capacity the maximum capacity for the {@link LRUMapCapacityBound}
*/
public LRUMapCapacityBound(int capacity) {
super(16, 0.75f, true);
this.cacheSize = capacity;
}
/**
* Initializes a new instance of the {@link LRUMapCapacityBound} class with the specified capacity and initial values
*
* @param capacity the maximum capacity for the {@link LRUMapCapacityBound}
* @param values initial values to populate the {@link LRUMapCapacityBound}
*/
public LRUMapCapacityBound(int capacity, Map<? extends K, ? extends V> values) {
this(capacity);
putAll(values);
}
@Override
protected boolean removeEldestEntry(Map.Entry<K, V> eldest) {
return size() > cacheSize;
}
}
binjr/binjr-core/src/main/java/eu/binjr/common/cache/LRUMapSizeBound.java
Vendored
+61
View File
@@ -0,0 +1,61 @@
/*
* Copyright 2017-2018 Frederic Thevenet
*
* Licensed under the Apache License, Version 2.0 (the "License");
* you may not use this file except in compliance with the License.
* You may obtain a copy of the License at
*
* http://www.apache.org/licenses/LICENSE-2.0
*
* Unless required by applicable law or agreed to in writing, software
* distributed under the License is distributed on an "AS IS" BASIS,
* WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
* See the License for the specific language governing permissions and
* limitations under the License.
*/
package eu.binjr.common.cache;
import java.util.LinkedHashMap;
import java.util.Map;
/**
* Map witch evicts the least recently used element when a maximum size of stored element is reached.
*
* <p><i><b>NB: </b>This implementation recalculates the combined size of all stored elements each time a new element is added,
* so it is not well suited to pushing a large number of cached items at a high rate.</i></p>
*
* @param <K> type of keys
* @param <V> type of values
* @author Frederic Thevenet
*/
@Deprecated
public class LRUMapSizeBound<K, V extends Cacheable> extends LinkedHashMap<K, V> {
private long maxSize;
/**
* Initializes a new instance of the {@link LRUMapSizeBound} class with the specified capacity
*
* @param maxSize the maximum capacity for the {@link LRUMapSizeBound}
*/
public LRUMapSizeBound(int maxSize) {
super(16, 0.75f, true);
this.maxSize = maxSize;
}
/**
* Initializes a new instance of the {@link LRUMapSizeBound} class with the specified capacity and initial values
*
* @param maxSize the maximum capacity for the {@link LRUMapSizeBound}
* @param values initial values to populate the {@link LRUMapSizeBound}
*/
public LRUMapSizeBound(int maxSize, Map<? extends K, ? extends V> values) {
this(maxSize);
putAll(values);
}
@Override
protected boolean removeEldestEntry(Map.Entry<K, V> eldest) {
return this.values().stream().map(Cacheable::getSize).reduce(0L, Long::sum) > maxSize;
}
}
binjr/binjr-core/src/main/java/eu/binjr/common/colors/ColorUtils.java
+90
View File
@@ -0,0 +1,90 @@
/*
* Copyright 2020 Frederic Thevenet
*
* Licensed under the Apache License, Version 2.0 (the "License");
* you may not use this file except in compliance with the License.
* You may obtain a copy of the License at
*
* http://www.apache.org/licenses/LICENSE-2.0
*
* Unless required by applicable law or agreed to in writing, software
* distributed under the License is distributed on an "AS IS" BASIS,
* WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
* See the License for the specific language governing permissions and
* limitations under the License.
*/
package eu.binjr.common.colors;
import eu.binjr.common.logging.Logger;
import javafx.scene.paint.Color;
/**
* Utility methods to convert color encodings
*
* @author Frederic Thevenet
*/
public class ColorUtils {
private static final Logger logger = Logger.create(ColorUtils.class);
public static String toRGBAString(Color color) {
return toRGBAString(color, color.getOpacity());
}
public static String toRGBAString(Color color, double alpha) {
if (color == null) {
throw new IllegalArgumentException("Argument color is null");
}
if (alpha > 1.0 || alpha < 0) {
throw new IllegalArgumentException("alpha parameter value is out of bound: " + alpha);
}
StringBuilder sb = new StringBuilder("rgba(")
.append(color.getRed())
.append(",")
.append(color.getGreen())
.append(",")
.append(color.getBlue())
.append(",")
.append(alpha)
.append(")");
logger.debug(() -> "RGBA representation = " + sb.toString());
return sb.toString();
}
public static String toHex(Color color, String defaultHexValue) {
if (defaultHexValue == null) {
throw new IllegalArgumentException("Default color hex value cannot be null");
}
if (color == null) {
return defaultHexValue;
}
return toHex(color);
}
public static String toHex(Color color) {
if (color == null) {
throw new IllegalArgumentException("Argument color is null");
}
return toHex(color, color.getOpacity());
}
public static String toHex(Color color, double alpha) {
if (color == null) {
throw new IllegalArgumentException("Argument color is null");
}
return String.format("#%02x%02x%02x%02x",
Math.round(color.getRed() * 255),
Math.round(color.getGreen() * 255),
Math.round(color.getBlue() * 255),
Math.round(alpha * 255));
}
public static Color copy(Color color){
return Color.color(color.getRed(),
color.getGreen(),
color.getBlue(),
color.getOpacity());
}
}
binjr/binjr-core/src/main/java/eu/binjr/common/concurrent/CloseableResourceManager.java
+216
View File
@@ -0,0 +1,216 @@
package eu.binjr.common.concurrent;
import org.apache.commons.logging.Log;
import org.apache.commons.logging.LogFactory;
import java.io.Closeable;
import java.util.HashMap;
import java.util.Map;
import java.util.Optional;
import java.util.concurrent.atomic.AtomicInteger;
import java.util.concurrent.locks.ReentrantReadWriteLock;
import java.util.function.Supplier;
/**
* A generic resource manager for safely initializing, sharing and disposing
* singletons amongst multiple consumer objects, on multiple threads.
* <p>
* The resource manager will guarantee that only one instance for each supplied
* key is ever is initialized and returned when required.
* </p>
* <p>
* Reference counting is used to ensure that the actual closing of a resource
* only happens once all registered consumers have released it.
* </p>
*
* @param <T> Resource type
* @author Frederic Thevenet
*/
public class CloseableResourceManager<T extends Closeable> {
private static final Log logger = LogFactory.getLog(CloseableResourceManager.class);
private final Map<String, ResourceHolder<T>> resources = new HashMap<>();
private final ReadWriteLockHelper managerLock = new ReadWriteLockHelper(new ReentrantReadWriteLock());
/**
* Resource holder Resource type
*
* @param <U> Resource type
*/
private final class ResourceHolder<U extends AutoCloseable> {
private AtomicInteger referenceCount = new AtomicInteger(0);
private volatile boolean closed;
private U instance;
/**
* Constructor
*
* @param instance Resource instance
*/
private ResourceHolder(U instance) {
this.instance = instance;
}
/**
* Get Resource instance
*
* @return Resource instance
*/
private U getInstance() {
return instance;
}
/**
* Register resource
*
* @return Reference count
*/
private int register() {
return referenceCount.incrementAndGet();
}
/**
* Release resource
*
* @return Reference count
* @throws Exception exception
*/
private int release() throws Exception {
int refLeft = referenceCount.decrementAndGet();
if (refLeft < 1) {
if (!closed) {
try {
instance.close();
} finally {
closed = true;
}
}
}
return refLeft;
}
}
/**
* Registers a consumer with a resource, identified by the supplied key.
* <p>
* This method must only be called once per consumer (or the release methods
* must be called accordingly), otherwise the resource may never be
* released.
* </p>
*
* @param key the key to identify the resource to register.
* @param resourceFactory a factory method used to build a new instance of the resource
* if one doesn't already exist.
* @return the registered resource
*/
public T acquire(String key, Supplier<T> resourceFactory) {
return managerLock.write().lock(() -> {
var r = resources.computeIfAbsent(key, k -> new ResourceHolder<>(resourceFactory.get()));
r.register();
return r.getInstance();
});
}
/**
* Registers a consumer with a resource, identified by the supplied key.
* <p>
* This method must only be called once per consumer (or the release methods
* must be called accordingly), otherwise the resource may never be
* released.
* </p>
*
* @param key the key to identify the resource to register.
* @param resource the resource to register
* @return Reference count
*/
public int register(String key, T resource) {
return managerLock.write().lock(() -> {
var r = resources.computeIfAbsent(key, k -> new ResourceHolder<>(resource));
return r.register();
});
}
/**
* Releases the resource identified by the supplied key.
* <p>
* If all references to this resource are released, the resource is closed.
* </p>
* <p>
* <b>Warning:</b> Registering a closed resources with the same key will
* cause a new instance to be created.
* </p>
*
* @param key the key to identify the resource to release.
* @return the number of references left for the specified resource.
* @throws Exception if an error occurs while closing the resource.
*/
public int release(String key) throws Exception {
return managerLock.write().lock(() -> {
ResourceHolder<T> r = resources.get(key);
if (r != null) {
try {
return r.release();
} finally {
if (r.closed) {
resources.remove(key);
}
}
} else {
logger.warn("Trying to release a resource that is not registered.");
return -1;
}
});
}
/**
* Forces the release and subsequent closing of the specified resource,
* regardless of how many references to it are still held by other
* consumers.
* <p>
* Use with caution.
* </p>
*
* @param key the key to identify the resource to close.
* @throws Exception if an error occurs while closing the resource.
*/
public void forceReleaseAndClose(String key) throws Exception {
managerLock.write().lock(() -> {
ResourceHolder<T> r = resources.get(key);
if (r != null) {
while (r.release() > 0) {
// Nothing
}
} else {
logger.warn("Trying to close a resource that is not registered.");
}
});
}
/**
* Returns the resource identified by the supplied key.
*
* @param key the key to identify the resource to release.
* @return An {@link Optional} wrapping the resource identified by the supplied key.
*/
public Optional<T> get(String key) {
return managerLock.read().lock(() -> {
ResourceHolder<T> r = resources.get(key);
if (r != null) {
return Optional.of(r.getInstance());
}
return Optional.empty();
});
}
/**
* Returns true if a resource identified by the provided key is registered,
* false otherwise.
*
* @param key the key to identify the resource.
* @return true if a resource identified by the provided key is registered,
* false otherwise.
*/
public boolean isRegistered(String key) {
return managerLock.read().lock(() -> resources.containsKey(key));
}
}
binjr/binjr-core/src/main/java/eu/binjr/common/concurrent/ReadWriteLockHelper.java
+306
View File
@@ -0,0 +1,306 @@
/*
* Copyright 2017-2018 Frederic Thevenet
*
* Licensed under the Apache License, Version 2.0 (the "License");
* you may not use this file except in compliance with the License.
* You may obtain a copy of the License at
*
* http://www.apache.org/licenses/LICENSE-2.0
*
* Unless required by applicable law or agreed to in writing, software
* distributed under the License is distributed on an "AS IS" BASIS,
* WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
* See the License for the specific language governing permissions and
* limitations under the License.
*/
package eu.binjr.common.concurrent;
import eu.binjr.common.function.*;
import java.util.Optional;
import java.util.concurrent.locks.Lock;
import java.util.concurrent.locks.ReadWriteLock;
import java.util.concurrent.locks.ReentrantReadWriteLock;
/**
* An helper class that wraps a {@link ReadWriteLock} instance and provides methods to streamline usage and avoid common inattention mistakes,
* such as locking one the lock and unlocking the other in the {@code finally} block.
*
* @author Frederic Thevenet
*/
public class ReadWriteLockHelper {
private final ReadWriteLock lock;
private final LockHelper readLockHelper;
private final LockHelper writeLockHelper;
/**
* Creates a new instance of the {@link ReadWriteLockHelper} class that wraps a new instance of {@link ReentrantReadWriteLock}
*/
public ReadWriteLockHelper() {
this(new ReentrantReadWriteLock());
}
/**
* Creates a new instance of the {@link ReadWriteLockHelper} class for the provided lock.
*
* @param lock the {@link ReadWriteLock} instance to wrap.
*/
public ReadWriteLockHelper(ReadWriteLock lock) {
this.lock = lock;
this.readLockHelper = new LockHelper(lock.readLock());
this.writeLockHelper = new LockHelper(lock.writeLock());
}
/**
* Gets the underlying ReadWriteLock instance.
*
* @return the underlying ReadWriteLock instance.
*/
public ReadWriteLock getLock() {
return lock;
}
/**
* Returns an helper providing methods to run lambdas within the context of the read lock.
*
* @return an helper providing methods to run lambdas within the context of the read lock.
*/
public LockHelper read() {
return readLockHelper;
}
/**
* Returns an helper providing methods to run lambdas within the context of the write lock.
*
* @return an helper providing methods to run lambdas within the context of the write lock.
*/
public LockHelper write() {
return writeLockHelper;
}
/**
* An helper providing methods to run lambdas within the context of supplied lock.
*/
public static class LockHelper {
private final Lock lock;
/**
* Constructor
*
* @param lock Lock
*/
private LockHelper(Lock lock) {
this.lock = lock;
}
//region *** lock method overloads ***
/**
* Encapsulated the evaluation of the supplied {@link CheckedSupplier} within the boundaries of the lock.
*
* @param operation the {@link CheckedSupplier} to evaluate.
* @param <R> the return type for the {@link CheckedSupplier}.
* @param <E> the type of the exception thrown by the {@link CheckedSupplier}.
* @return the return of the {@link CheckedSupplier}.
* @throws E the exception thrown by the {@link CheckedSupplier}.
*/
public <R, E extends Exception> R lock(CheckedSupplier<R, E> operation) throws E {
lock.lock();
try {
return operation.get();
} finally {
lock.unlock();
}
}
/**
* Encapsulated the evaluation of the supplied {@link CheckedFunction} within the boundaries of the lock.
*
* @param operation the {@link CheckedFunction} to apply.
* @param t the parameter to pass the {@link CheckedFunction}
* @param <T> the type of he parameter to pass the {@link CheckedFunction}
* @param <R> the return type for the {@link CheckedFunction}.
* @param <E> the type of the exception thrown by the {@link CheckedFunction}.
* @return the result of the {@link CheckedFunction}.
* @throws E the exception thrown by the {@link CheckedFunction}.
*/
public <T, R, E extends Exception> R lock(CheckedFunction<T, R, E> operation, T t) throws E {
lock.lock();
try {
return operation.apply(t);
} finally {
lock.unlock();
}
}
/**
* Encapsulated the evaluation of the supplied {@link CheckedBiFunction} within the boundaries of the lock.
*
* @param operation the {@link CheckedBiFunction} to apply.
* @param t the first parameter to pass the {@link CheckedBiFunction}.
* @param u the second parameter to pass the {@link CheckedBiFunction}.
* @param <T> the type of the first parameter of the {@link CheckedBiFunction}.
* @param <U> the type of the first parameter of the {@link CheckedBiFunction}.
* @param <R> the return type for the {@link CheckedBiFunction}.
* @param <E> the type of the exception thrown by the {@link CheckedBiFunction}.
* @return the result of the {@link CheckedFunction}.
* @throws E the exception thrown by the {@link CheckedBiFunction}.
*/
public <T, U, R, E extends Exception> R lock(CheckedBiFunction<T, U, R, E> operation, T t, U u) throws E {
lock.lock();
try {
return operation.apply(t, u);
} finally {
lock.unlock();
}
}
/**
* Encapsulated the evaluation of the supplied {@link CheckedRunnable} within the boundaries of the lock.
*
* @param operation the {@link CheckedRunnable} to evaluate.
* @param <E> the type of the exception thrown by the {@link CheckedRunnable}.
* @throws E the exception thrown by the {@link CheckedRunnable}.
*/
public <E extends Exception> void lock(CheckedRunnable<E> operation) throws E {
lock.lock();
try {
operation.run();
} finally {
lock.unlock();
}
}
/**
* Encapsulated the evaluation of the supplied {@link CheckedConsumer} within the boundaries of the lock.
*
* @param operation the {@link CheckedConsumer} to evaluate.
* @param t the parameter to pass the {@link CheckedConsumer}
* @param <T> the type of the parameter to pass the {@link CheckedConsumer}
* @param <E> the type of the exception thrown by the {@link CheckedConsumer}.
* @throws E the exception thrown by the {@link CheckedConsumer}.
*/
public <T, E extends Exception> void lock(CheckedConsumer<T, E> operation, T t) throws E {
lock.lock();
try {
operation.accept(t);
} finally {
lock.unlock();
}
}
//endregion
//region *** tryLock method overloads ***
/**
* Tries to acquire the lock and if successful, evaluates the supplied {@link CheckedSupplier} within the boundaries of the lock.
*
* @param operation the {@link CheckedSupplier} to evaluate.
* @param <R> the return type for the {@link CheckedSupplier}.
* @param <E> the type of the exception thrown by the {@link CheckedSupplier}.
* @return An {@link Optional} that contains the return value of the {@link CheckedSupplier} if the lock could be acquired
* @throws E the exception thrown by the {@link CheckedSupplier}.
*/
public <R, E extends Exception> Optional<R> tryLock(CheckedSupplier<R, E> operation) throws E {
if (lock.tryLock()) {
try {
return Optional.ofNullable(operation.get());
} finally {
lock.unlock();
}
}
return Optional.empty();
}
/**
* Tries to acquire the lock and if successful, evaluates the supplied {@link CheckedFunction} within the boundaries of the lock.
*
* @param operation the {@link CheckedFunction} to apply.
* @param t the parameter to pass the {@link CheckedFunction}
* @param <T> the type of he parameter to pass the {@link CheckedFunction}
* @param <R> the return type for the {@link CheckedFunction}.
* @param <E> the type of the exception thrown by the {@link CheckedFunction}.
* @return the result of the {@link CheckedFunction}.
* @throws E the exception thrown by the {@link CheckedFunction}.
*/
public <T, R, E extends Exception> Optional<R> tryLock(CheckedFunction<T, R, E> operation, T t) throws E {
if (lock.tryLock()) {
try {
return Optional.ofNullable(operation.apply(t));
} finally {
lock.unlock();
}
}
return Optional.empty();
}
/**
* Tries to acquire the lock and if successful, evaluates the supplied {@link CheckedBiFunction} within the boundaries of the lock.
*
* @param operation the {@link CheckedBiFunction} to apply.
* @param t the first parameter to pass the {@link CheckedBiFunction}.
* @param u the second parameter to pass the {@link CheckedBiFunction}.
* @param <T> the type of the first parameter of the {@link CheckedBiFunction}.
* @param <U> the type of the first parameter of the {@link CheckedBiFunction}.
* @param <R> the return type for the {@link CheckedBiFunction}.
* @param <E> the type of the exception thrown by the {@link CheckedBiFunction}.
* @return the result of the {@link CheckedFunction}.
* @throws E the exception thrown by the {@link CheckedBiFunction}.
*/
public <T, U, R, E extends Exception> Optional<R> tryLock(CheckedBiFunction<T, U, R, E> operation, T t, U u) throws E {
if (lock.tryLock()) {
try {
return Optional.ofNullable(operation.apply(t, u));
} finally {
lock.unlock();
}
}
return Optional.empty();
}
/**
* Tries to acquire the lock and if successful, evaluates the supplied {@link CheckedRunnable} within the boundaries of the lock.
*
* @param operation the {@link CheckedRunnable} to evaluate.
* @param <E> the type of the exception thrown by the {@link CheckedRunnable}.
* @return true if the lock was acquired, false otherwise.
* @throws E the exception thrown by the {@link CheckedRunnable}.
*/
public <E extends Exception> boolean tryLock(CheckedRunnable<E> operation) throws E {
if (lock.tryLock()) {
try {
operation.run();
} finally {
lock.unlock();
}
return true;
}
return false;
}
/**
* Tries to acquire the lock and if successful, evaluates the supplied {@link CheckedConsumer} within the boundaries of the lock.
*
* @param operation the {@link CheckedConsumer} to evaluate.
* @param t the parameter to pass the {@link CheckedConsumer}
* @param <T> the type of the parameter to pass the {@link CheckedConsumer}
* @param <E> the type of the exception thrown by the {@link CheckedConsumer}.
* @return true if the lock was acquired, false otherwise.
* @throws E the exception thrown by the {@link CheckedConsumer}.
*/
public <T, E extends Exception> boolean tryLock(CheckedConsumer<T, E> operation, T t) throws E {
if (lock.tryLock()) {
try {
operation.accept(t);
} finally {
lock.unlock();
}
return true;
}
return false;
}
//endregion
}
}
binjr/binjr-core/src/main/java/eu/binjr/common/diagnostic/DiagnosticCommand.java
+175
View File
@@ -0,0 +1,175 @@
/*
* Copyright 2017-2020 Frederic Thevenet
*
* Licensed under the Apache License, Version 2.0 (the "License");
* you may not use this file except in compliance with the License.
* You may obtain a copy of the License at
*
* http://www.apache.org/licenses/LICENSE-2.0
*
* Unless required by applicable law or agreed to in writing, software
* distributed under the License is distributed on an "AS IS" BASIS,
* WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
* See the License for the specific language governing permissions and
* limitations under the License.
*/
package eu.binjr.common.diagnostic;
import eu.binjr.core.preferences.AppEnvironment;
import javax.management.JMX;
import javax.management.MBeanServer;
import javax.management.MalformedObjectNameException;
import javax.management.ObjectName;
import java.lang.management.ManagementFactory;
import java.nio.file.Path;
import java.util.function.Supplier;
/**
* Defines methods for a diagnostic command.
*/
public interface DiagnosticCommand {
DiagnosticCommand local = ((Supplier<DiagnosticCommand>) () -> {
try {
MBeanServer server = ManagementFactory.getPlatformMBeanServer();
ObjectName name = new ObjectName("com.sun.management", "type", "DiagnosticCommand");
return JMX.newMBeanProxy(server, name, DiagnosticCommand.class);
} catch (MalformedObjectNameException e) {
throw new AssertionError(e);
}
}).get();
static String dumpVmSystemProperties() throws DiagnosticException {
try {
return local.vmSystemProperties();
} catch (Throwable t) {
throw new DiagnosticException("Failed to invoke diagnostic command VM.System_properties", t);
}
}
static String dumpThreadStacks() throws DiagnosticException {
try {
return local.threadPrint("-l");
} catch (Throwable t) {
throw new DiagnosticException("Failed to invoke diagnostic command Thread.print", t);
}
}
static String dumpClassHistogram() throws DiagnosticException {
try {
return local.gcClassHistogram();
} catch (Throwable t) {
throw new DiagnosticException("Failed to invoke diagnostic command GC.class_histogram", t);
}
}
static String dumpVmFlags() throws DiagnosticException {
try {
return local.vmFlags();
} catch (Throwable t) {
throw new DiagnosticException("Failed to invoke diagnostic command VM.flags", t);
}
}
static String dumpVmCommandLine() throws DiagnosticException {
try {
return local.vmCommandLine();
} catch (Throwable t) {
throw new DiagnosticException("Failed to invoke diagnostic command VM.command_line", t);
}
}
static String getHelp() throws DiagnosticException {
try {
return local.help();
} catch (Throwable t) {
throw new DiagnosticException("Failed to invoke diagnostic command help", t);
}
}
static void dumpHeap(Path path) throws DiagnosticException {
switch (AppEnvironment.getInstance().getRunningJvm()) {
case HOTSPOT:
HotSpotDiagnosticHelper.dumpHeap(path);
break;
case OPENJ9:
case UNSUPPORTED:
default:
throw new UnavailableDiagnosticFeatureException();
}
}
static Path getHeapDumpPath() throws DiagnosticException {
switch (AppEnvironment.getInstance().getRunningJvm()) {
case HOTSPOT:
return HotSpotDiagnosticHelper.getHeapDumpPath();
case OPENJ9:
case UNSUPPORTED:
default:
throw new UnavailableDiagnosticFeatureException();
}
}
static void setHeapDumpPath(Path path) throws DiagnosticException {
switch (AppEnvironment.getInstance().getRunningJvm()) {
case HOTSPOT:
HotSpotDiagnosticHelper.setHeapDumpPath(path);
break;
case OPENJ9:
case UNSUPPORTED:
default:
throw new UnavailableDiagnosticFeatureException();
}
}
static boolean getHeapDumpOnOutOfMemoryError() throws DiagnosticException {
switch (AppEnvironment.getInstance().getRunningJvm()) {
case HOTSPOT:
return HotSpotDiagnosticHelper.getHeapDumpOnOutOfMemoryError();
case OPENJ9:
case UNSUPPORTED:
default:
throw new UnavailableDiagnosticFeatureException();
}
}
static void setHeapDumpOnOutOfMemoryError(boolean value) throws DiagnosticException {
switch (AppEnvironment.getInstance().getRunningJvm()) {
case HOTSPOT:
HotSpotDiagnosticHelper.setHeapDumpOnOutOfMemoryError(value);
break;
case OPENJ9:
case UNSUPPORTED:
default:
throw new UnavailableDiagnosticFeatureException();
}
}
static String dumpVmOptions() throws DiagnosticException {
switch (AppEnvironment.getInstance().getRunningJvm()) {
case HOTSPOT:
return HotSpotDiagnosticHelper.dumpVmOptions();
case OPENJ9:
case UNSUPPORTED:
default:
throw new UnavailableDiagnosticFeatureException();
}
}
String threadPrint(String... args);
String help(String... args);
String vmSystemProperties(String... args);
String gcClassHistogram(String... args);
String vmFlags(String... args);
String vmCommandLine(String... args);
}
binjr/binjr-core/src/main/java/eu/binjr/common/diagnostic/DiagnosticException.java
+59
View File
@@ -0,0 +1,59 @@
/*
* Copyright 2017-2018 Frederic Thevenet
*
* Licensed under the Apache License, Version 2.0 (the "License");
* you may not use this file except in compliance with the License.
* You may obtain a copy of the License at
*
* http://www.apache.org/licenses/LICENSE-2.0
*
* Unless required by applicable law or agreed to in writing, software
* distributed under the License is distributed on an "AS IS" BASIS,
* WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
* See the License for the specific language governing permissions and
* limitations under the License.
*/
package eu.binjr.common.diagnostic;
/**
* Signals that an error occurred while invoking a diagnostic command.
*
* @author Frederic Thevenet
*/
public class DiagnosticException extends Exception {
/**
* Creates a new instance of the {@link DiagnosticException} class.
*/
public DiagnosticException() {
super("An error occurred while invoking a diagnostic command.");
}
/**
* Creates a new instance of the {@link DiagnosticException} class with the provided message.
*
* @param message the message of the exception.
*/
public DiagnosticException(String message) {
super(message);
}
/**
* Creates a new instance of the {@link DiagnosticException} class with the provided message and cause {@link Throwable}
*
* @param message the message of the exception.
* @param cause the cause for the exception.
*/
public DiagnosticException(String message, Throwable cause) {
super(message, cause);
}
/**
* Creates a new instance of the {@link DiagnosticException} class with the provided cause {@link Throwable}
*
* @param cause the cause for the exception.
*/
public DiagnosticException(Throwable cause) {
super(cause);
}
}
binjr/binjr-core/src/main/java/eu/binjr/common/diagnostic/HotSpotDiagnosticHelper.java
+91
View File
@@ -0,0 +1,91 @@
/*
* Copyright 2019-2020 Frederic Thevenet
*
* Licensed under the Apache License, Version 2.0 (the "License");
* you may not use this file except in compliance with the License.
* You may obtain a copy of the License at
*
* http://www.apache.org/licenses/LICENSE-2.0
*
* Unless required by applicable law or agreed to in writing, software
* distributed under the License is distributed on an "AS IS" BASIS,
* WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
* See the License for the specific language governing permissions and
* limitations under the License.
*/
package eu.binjr.common.diagnostic;
import com.sun.management.HotSpotDiagnosticMXBean;
import com.sun.management.VMOption;
import java.lang.management.ManagementFactory;
import java.nio.file.Path;
import java.util.List;
import java.util.stream.Collectors;
/**
* Defines methods for HotSpot specific diagnistic commands.
*/
public interface HotSpotDiagnosticHelper {
HotSpotDiagnosticMXBean HOTSPOT_DIAGNOSTIC = ManagementFactory.getPlatformMXBean(HotSpotDiagnosticMXBean.class);
static String dumpVmOptions() throws DiagnosticException{
return "Hotspot VM Options\n" + listOption()
.stream()
.map(vmOption -> String.format("%s=%s (%s)",
vmOption.getName(),
vmOption.getValue(),
vmOption.getOrigin()))
.collect(Collectors.joining("\n"));
}
static List<VMOption> listOption() throws DiagnosticException {
try {
return HOTSPOT_DIAGNOSTIC.getDiagnosticOptions();
} catch (Throwable t) {
throw new DiagnosticException("Failed to list VMOptions", t);
}
}
static void dumpHeap(Path path) throws DiagnosticException {
try {
HOTSPOT_DIAGNOSTIC.dumpHeap(path.toString(), false);
} catch (Throwable t) {
throw new DiagnosticException("Failed to get VMOption HeapDumpPath", t);
}
}
static Path getHeapDumpPath() throws DiagnosticException {
try {
return Path.of(HOTSPOT_DIAGNOSTIC.getVMOption("HeapDumpPath").getValue());
} catch (Throwable t) {
throw new DiagnosticException("Failed to get VMOption HeapDumpPath", t);
}
}
static void setHeapDumpPath(Path path) throws DiagnosticException {
try {
HOTSPOT_DIAGNOSTIC.setVMOption("HeapDumpPath", path.toString());
} catch (Throwable t) {
throw new DiagnosticException("Failed to set VMOption HeapDumpPath", t);
}
}
static boolean getHeapDumpOnOutOfMemoryError() throws DiagnosticException {
try {
return Boolean.parseBoolean(HOTSPOT_DIAGNOSTIC.getVMOption("HeapDumpOnOutOfMemoryError").getValue());
} catch (Throwable t) {
throw new DiagnosticException("Failed to get VMOption HeapDumpOnOutOfMemoryError", t);
}
}
static void setHeapDumpOnOutOfMemoryError(boolean value) throws DiagnosticException {
try {
HOTSPOT_DIAGNOSTIC.setVMOption("HeapDumpOnOutOfMemoryError", Boolean.toString(value));
} catch (Throwable t) {
throw new DiagnosticException("Failed to set VMOption HeapDumpOnOutOfMemoryError", t);
}
}
}

Some files were not shown because too many files have changed in this diff Show More