[gentoo-commits] proj/gentoo-mirrorstats:master commit in: json/, /

[Max Magorsch]

commit:     92ea4ca1710dd5eefbda4f97687ea76e34812d50
Author:     Max Magorsch <arzano <AT> gentoo <DOT> org>
AuthorDate: Mon Apr 27 18:08:22 2020 +0000
Commit:     Max Magorsch <arzano <AT> gentoo <DOT> org>
CommitDate: Mon Apr 27 18:08:22 2020 +0000
URL:        https://gitweb.gentoo.org/proj/gentoo-mirrorstats.git/commit/?id=92ea4ca1

Generate a json file containing the mirmon data

The new generate-json.py script will parse the data
generated by mirmon which is present in the state
file and convert it to json. A json file called
'state.json' will be created so that the mirmon
data can be properly parsed.

Signed-off-by: Max Magorsch <arzano <AT> gentoo.org>

 json/generate-json.py | 47 +++++++++++++++++++++++++++++++++++++++++++++++
 mirmon-distfiles.sh   |  4 +++-
 mirmon-rsync.sh       |  4 +++-
 3 files changed, 53 insertions(+), 2 deletions(-)

diff --git a/json/generate-json.py b/json/generate-json.py
new file mode 100755
index 0000000..1786bf9
--- /dev/null
+++ b/json/generate-json.py
@@ -0,0 +1,47 @@
+#!/usr/bin/python
+
+#
+# Convert a mirmon.state file to json
+#
+# This script will parse a mirmon state file, convert the content
+# to json and print the json to stdout.
+#
+# Usage:
+#   $ generate-json.py path_to_mirmon_state_file
+#
+# The json will be printed to stdout. In case of an error, the
+# exception will be printed to stderr.
+#
+
+import json
+import sys
+
+try:
+    with open(str(sys.argv[1])) as file_in:
+        mirrors = {}
+        for raw_line in file_in:
+
+            line = raw_line.split(" ")
+            protocol = line[0].split("://")[0]
+            url = line[0].split("://")[1]
+
+            state = {
+                "Protocol" : protocol,
+                "Url" : url,
+                "Age" : line[1],
+                "StatusLastProbe" : line[2],
+                "TimeLastSuccessfulProbe" : line[3],
+                "ProbeHistory" : line[4],
+                "StateHistory" : line[5],
+                "LastProbe" : line[6].strip(),
+            }
+
+            if url not in mirrors:
+                mirrors[url] = []
+
+            mirrors[url].append(state)
+
+        print(json.dumps(mirrors))
+
+except Exception as e:
+    print("Could not convert mirmon state file to json: " + str(e), file=sys.stderr)

diff --git a/mirmon-distfiles.sh b/mirmon-distfiles.sh
index f8012cc..3e14e87 100755
--- a/mirmon-distfiles.sh
+++ b/mirmon-distfiles.sh
@@ -13,4 +13,6 @@ cd /var/www/mirrorstats.gentoo.org/gentoo-mirrorstats/distfiles_mirrors
 sed \
     -e 's#mirrors</H2>#<a href="http://www.gentoo.org/main/en/mirrors2.xml">mirrors</a></H2>#' \
     >../../htdocs/distfiles/index.html <../../htdocs/distfiles/index-wip.html
-
+# Generate a json file containing the state of each mirror
+../json/generate-json.py /var/www/mirrorstats.gentoo.org/gentoo-mirrorstats/distfiles_mirrors/var/mirmon.state \
+  > ../../htdocs/distfiles/state.json

diff --git a/mirmon-rsync.sh b/mirmon-rsync.sh
index 5df1c73..900832d 100755
--- a/mirmon-rsync.sh
+++ b/mirmon-rsync.sh
@@ -28,4 +28,6 @@ echo "gentoo rsync://35.190.132.250" >> ./var/g.mirrors
 sed \
     -e 's#mirrors</H2>#<a href="http://www.gentoo.org/main/en/mirrors-rsync.xml">mirrors</a></H2>#' \
     >../../htdocs/rsync/index.html <../../htdocs/rsync/index-wip.html
-
+# Generate a json file containing the state of each mirror
+../json/generate-json.py /var/www/mirrorstats.gentoo.org/gentoo-mirrorstats/rsync_mirrors/var/mirmon.state \
+  > ../../htdocs/rsync/state.json