www/caddy: Add Caddy widgets for 24.7 new dashboard (#4049)

* Added Caddy widget that shows managed domains with their enabled status in the new Dashboard.

* Prepare new API endpoint for widget to display certificate validity for the automatic certificates. Swings magic wand ~asyncio.

* Add new widget to show validity status of Caddy certificates, consuming the new diagnostics certificate API.

* Missed endpoint in CaddyCertificate widget.

* Bump plugin version to 1.6.0 for 24.7

* Sort certificates by expiration date, add Expires: before the date for more clarity.

* Change cache_ttl to half a minute and fix style.

* Improve error handling of CaddyCertificate widget and API.

* Display both domains and subdomains in CaddyDomain widget. Allow free resize since the list can be long.

* Improve CaddyCertificate widget. API now serves remaining_days. Tooltip turns yellow when 14 days are left. The remaining days are shown in the overview next to the date.

* Add changelog for fix in other PR.

* Change font awesome to lock and globe symbols.

* Improve error handling of tasks with asyncio.wait_for() and a timeout.

* Update CaddyDomain and CaddyCertificate widget to emply caching of data, to only update the UI if changes are detected in the data.

* Generalize dataHasChange method in CaddyCertificate widget to compare strings.

* Implement error utility function, some cleanup for consistency.

* Update pkg-descr - Add changelog from other PRs

* Remove indent from json.dumps.

* Add widget metadata.

* Change tick timeout since resize issues have been fixed.

* Update pkg-descr

* Update pkg-descr

* Remove dataHasChanged in favor of generalized dataChanged of Base Widget.

* Update pkg-descr

www/caddy/Makefile

@@ -1,5 +1,5 @@
 PLUGIN_NAME=		caddy
-PLUGIN_VERSION=		1.5.7
+PLUGIN_VERSION=		1.6.0
 PLUGIN_DEPENDS=		caddy-custom
 PLUGIN_COMMENT=		Easy to configure Reverse Proxy with Automatic HTTPS and Dynamic DNS
 PLUGIN_MAINTAINER=	cedrik@pischem.com

www/caddy/pkg-descr

@@ -17,7 +17,6 @@ Main features of this plugin:
 * Access Lists to restrict access based on static networks
 * Basic Auth to restrict access by username and password
 * Syslog-ng integration and HTTP Access Log
-* NTLM Transport
 * Header manipulation with header_up and header_down
 * Simple load balancing with passive health check
 
@@ -26,6 +25,16 @@ DOC: https://docs.opnsense.org/manual/how-tos/caddy.html
 Plugin Changelog
 ================
 
+1.6.0
+
+* Add: New Dashboard widgets for 24.7, showing domain status and certificate validity status.
+* Fix: Caddyfile template fixed for IPv6 addresses and custom port.
+* Add: forward_auth directive with Authelia as Authz Provider.
+* Add: Default HTTP and HTTPS ports can be changed in general settings.
+* Add: Introduce HTTP version to handler. HTTP/1.1, HTTP/2 and HTTP/3 can be chosen.
+* Change: NTLM is now deprecated, though the option will stay for a while longer.
+* Change: Option "tls_trusted_ca_certs" is now "tls_trust_pool".
+
 1.5.7
 
 * Build: Update to Caddy v2.8.4 + caddy-dns plugins updated to latest upstream versions

www/caddy/src/opnsense/mvc/app/controllers/OPNsense/Caddy/Api/DiagnosticsController.php

@@ -82,4 +82,24 @@ class DiagnosticsController extends ApiMutableModelControllerBase
         // Return the response as an array which gets automatically encoded to JSON
         return ["status" => "success", "content" => $responseArray['content']];
     }
+
+    /**
+     * Fetch the hostnames, validity and expiration dates of automatic certificates as JSON. Consumed by Caddy widget.
+     */
+    public function certificateAction()
+    {
+        $backend = new Backend();
+        $response = $backend->configdRun('caddy certificate');
+
+        // Decode JSON to PHP array
+        $responseArray = json_decode($response, true);
+
+        // Since errors are handled by the caddy_diagnostics script and returned as json, check for an error key in the response
+        if (isset($responseArray['error'])) {
+            return ["status" => "failed", "message" => $responseArray['message']];
+        }
+
+        // Return the response as an array which gets automatically encoded to JSON
+        return ["status" => "success", "content" => $responseArray];
+    }
 }

www/caddy/src/opnsense/scripts/OPNsense/Caddy/caddy_diagnostics.py

@@ -28,6 +28,10 @@
 
 import sys
 import json
+import os
+import subprocess
+import asyncio
+from datetime import datetime
 
 # Function to show the Caddy configuration from a JSON file
 def show_caddy_config():
@@ -61,12 +65,83 @@ def show_caddyfile():
     except Exception as e:
         print(json.dumps({"error": "General Error", "message": str(e)}))
 
+# Function to extract certificate information using openssl command
+async def extract_certificate_info(cert_path):
+    try:
+        # Execute the openssl command to get the expiration date with a timeout
+        result = await asyncio.wait_for(
+            asyncio.create_subprocess_exec(
+                'openssl', 'x509', '-in', cert_path, '-noout', '-enddate',
+                stdout=subprocess.PIPE, stderr=subprocess.PIPE),
+            timeout=10)  # Make sure tasks are cleaned up if they hang
+
+        stdout, stderr = await result.communicate()
+
+        # Check for errors in the execution
+        if result.returncode != 0:
+            error_message = stderr.decode().strip()
+            raise RuntimeError(f"Subprocess failed with error: {error_message}")
+
+        # Decode output and process the information
+        expiration_date_str = stdout.decode().strip().split('=')[1]
+
+        # Convert expiration date string to datetime object
+        expiration_date = datetime.strptime(expiration_date_str, "%b %d %H:%M:%S %Y GMT")
+
+        # Determine the current date
+        now = datetime.now()
+
+        # Calculate remaining days until expiration
+        remaining_days = (expiration_date - now).days
+        remaining_days = max(remaining_days, 0)  # Ensure non-negative days
+
+        # Extract the hostname from the filename
+        hostname = os.path.basename(cert_path).replace('.crt', '').lower()
+
+        return {'hostname': hostname, 'expiration_date': expiration_date_str, 'remaining_days': remaining_days}
+    except asyncio.TimeoutError as e:
+        # Handle timeout specific errors
+        raise RuntimeError(f"Timeout occurred while processing {cert_path}: {str(e)}")
+    except Exception as e:
+        raise RuntimeError(f"Error extracting certificate info for {cert_path}: {str(e)}")
+
+# Function to find certificates and create tasks to extract info
+async def find_certificates(base_dir):
+    tasks = []
+    for root, dirs, files in os.walk(base_dir):
+        # Skip any directories named 'temp'
+        dirs[:] = [d for d in dirs if d != 'temp']
+        for file in files:
+            if file.endswith('.crt'):
+                cert_path = os.path.join(root, file)
+                task = asyncio.create_task(extract_certificate_info(cert_path))
+                tasks.append(task)
+
+    if not tasks:
+        raise RuntimeError("No certificates were found in the specified directory.")
+
+    results = await asyncio.gather(*tasks, return_exceptions=True)
+    return [result for result in results if not isinstance(result, Exception)]
+
+# Function to show certificates, processing all found in the given directory
+async def show_certificates():
+    # Function to show certificates, processing all found in the given directory
+    base_dir = '/var/db/caddy/data/caddy/certificates'
+    try:
+        certificates_data = await find_certificates(base_dir)
+        if certificates_data:
+            print(json.dumps(certificates_data))
+        else:
+            raise RuntimeError("No valid certificate data found.")
+    except Exception as e:
+        print(json.dumps({"error": "General Error", "message": str(e)}))
+
 # Action handler
 def perform_action(action):
     actions = {
         "config": show_caddy_config,
-        "caddyfile": show_caddyfile
-        # Additional actions can be added here in the same format.
+        "caddyfile": show_caddyfile,
+        "certificate": lambda: asyncio.run(show_certificates())
     }
 
     action_func = actions.get(action)

www/caddy/src/opnsense/service/conf/actions.d/actions_caddy.conf

@@ -47,3 +47,9 @@ command:/usr/local/opnsense/scripts/OPNsense/Caddy/caddy_diagnostics.py caddyfil
 parameters:
 type:script_output
 message:Request Caddyfile
+
+[certificate]
+command:/usr/local/opnsense/scripts/OPNsense/Caddy/caddy_diagnostics.py certificate
+parameters:
+type:script_output
+message:Check validity of automatic certificates

www/caddy/src/opnsense/www/js/widgets/CaddyCertificate.js

@@ -0,0 +1,121 @@
+/*
+ * Copyright (C) 2024 Cedrik Pischem
+ * All rights reserved.
+ *
+ * Redistribution and use in source and binary forms, with or without
+ * modification, are permitted provided that the following conditions are met:
+ *
+ * 1. Redistributions of source code must retain the above copyright notice,
+ *    this list of conditions and the following disclaimer.
+ *
+ * 2. Redistributions in binary form must reproduce the above copyright
+ *    notice, this list of conditions and the following disclaimer in the
+ *    documentation and/or other materials provided with the distribution.
+ *
+ * THIS SOFTWARE IS PROVIDED ``AS IS'' AND ANY EXPRESS OR IMPLIED WARRANTIES,
+ * INCLUDING, BUT NOT LIMITED TO, THE IMPLIED WARRANTIES OF MERCHANTABILITY
+ * AND FITNESS FOR A PARTICULAR PURPOSE ARE DISCLAIMED. IN NO EVENT SHALL THE
+ * AUTHOR BE LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL, SPECIAL, EXEMPLARY,
+ * OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT LIMITED TO, PROCUREMENT OF
+ * SUBSTITUTE GOODS OR SERVICES; LOSS OF USE, DATA, OR PROFITS; OR BUSINESS
+ * INTERRUPTION) HOWEVER CAUSED AND ON ANY THEORY OF LIABILITY, WHETHER IN
+ * CONTRACT, STRICT LIABILITY, OR TORT (INCLUDING NEGLIGENCE OR OTHERWISE)
+ * ARISING IN ANY WAY OUT OF THE USE OF THIS SOFTWARE, EVEN IF ADVISED OF THE
+ * POSSIBILITY OF SUCH DAMAGE.
+ */
+
+import BaseTableWidget from "./BaseTableWidget.js";
+
+export default class CaddyCertificate extends BaseTableWidget {
+    constructor() {
+        super();
+        this.resizeHandles = "e, w";
+        this.tickTimeout = 30000;
+    }
+
+    getGridOptions() {
+        return {
+            // trigger overflow-y:scroll after 650px height
+            sizeToContent: 650
+        };
+    }
+
+    getMarkup() {
+        let $container = $('<div></div>');
+        let $caddyCertificateTable = this.createTable('caddyCertificateTable', {
+            headerPosition: 'none'
+        });
+
+        $container.append($caddyCertificateTable);
+        return $container;
+    }
+
+    async onWidgetTick() {
+        try {
+            // Check if Caddy is enabled
+            const caddyStatus = await ajaxGet('/api/caddy/reverse_proxy/get', {});
+            if (!caddyStatus.caddy.general || caddyStatus.caddy.general.enabled === "0") {
+                this.displayError(`${this.translations.unconfigured}`);
+                return;
+            }
+
+            // Fetch the certificate details
+            const response = await ajaxGet('/api/caddy/diagnostics/certificate', {});
+            if (response.status !== "success") {
+                this.displayError(`${this.translations.nocerts}`);
+                return;
+            }
+
+            // Process certificates if the response is successful
+            this.processCertificates(response.content);
+        } catch (error) {
+            this.displayError(`${this.translations.error}`);
+        }
+    }
+
+    // Utility function to display errors within the widget
+    displayError(message) {
+        const $error = $(`<div class="error-message"><a href="/ui/caddy/general">${message}</a></div>`);
+        $('#caddyCertificateTable').empty().append($error);
+    }
+
+    processCertificates(certificates) {
+        if (!this.dataChanged('certificates', certificates)) {
+            return;
+        }
+
+        let rows = certificates.map(certificate => {
+            let colorClass = 'text-success';
+            if (certificate.remaining_days === 0) {
+                colorClass = 'text-danger';
+            } else if (certificate.remaining_days < 14) {
+                colorClass = 'text-warning';
+            }
+
+            let statusText = certificate.remaining_days === 0 ? this.translations.expired :
+                             this.translations.valid;
+
+            let row = `
+                <div>
+                    <i class="fa fa-lock ${colorClass}" style="cursor: pointer;"
+                        data-toggle="tooltip" title="${statusText}">
+                    </i>
+                    &nbsp;
+                    <span><b>${certificate.hostname}</b></span>
+                    <br/>
+                    <div style="margin-top: 5px; margin-bottom: 5px;"><i>${this.translations.expires}</i> ${certificate.remaining_days} ${this.translations.days}, ${new Date(certificate.expiration_date).toLocaleString()}</div>
+                </div>`;
+            return { html: row, expirationDate: new Date(certificate.expiration_date) };
+        });
+
+        // Sort rows by expiration date from lowest to highest
+        rows.sort((a, b) => a.expirationDate - b.expirationDate);
+
+        // Extract sorted HTML rows and update table
+        let sortedRows = rows.map(row => [row.html]);
+        super.updateTable('caddyCertificateTable', sortedRows);
+
+        // Initialize tooltips for new elements
+        $('[data-toggle="tooltip"]').tooltip();
+    }
+}

www/caddy/src/opnsense/www/js/widgets/CaddyDomain.js

@@ -0,0 +1,119 @@
+/*
+ * Copyright (C) 2024 Cedrik Pischem
+ * All rights reserved.
+ *
+ * Redistribution and use in source and binary forms, with or without
+ * modification, are permitted provided that the following conditions are met:
+ *
+ * 1. Redistributions of source code must retain the above copyright notice,
+ *    this list of conditions and the following disclaimer.
+ *
+ * 2. Redistributions in binary form must reproduce the above copyright
+ *    notice, this list of conditions and the following disclaimer in the
+ *    documentation and/or other materials provided with the distribution.
+ *
+ * THIS SOFTWARE IS PROVIDED ``AS IS'' AND ANY EXPRESS OR IMPLIED WARRANTIES,
+ * INCLUDING, BUT NOT LIMITED TO, THE IMPLIED WARRANTIES OF MERCHANTABILITY
+ * AND FITNESS FOR A PARTICULAR PURPOSE ARE DISCLAIMED. IN NO EVENT SHALL THE
+ * AUTHOR BE LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL, SPECIAL, EXEMPLARY,
+ * OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT LIMITED TO, PROCUREMENT OF
+ * SUBSTITUTE GOODS OR SERVICES; LOSS OF USE, DATA, OR PROFITS; OR BUSINESS
+ * INTERRUPTION) HOWEVER CAUSED AND ON ANY THEORY OF LIABILITY, WHETHER IN
+ * CONTRACT, STRICT LIABILITY, OR TORT (INCLUDING NEGLIGENCE OR OTHERWISE)
+ * ARISING IN ANY WAY OUT OF THE USE OF THIS SOFTWARE, EVEN IF ADVISED OF THE
+ * POSSIBILITY OF SUCH DAMAGE.
+ */
+
+import BaseTableWidget from "./BaseTableWidget.js";
+
+export default class CaddyDomain extends BaseTableWidget {
+    constructor() {
+        super();
+        this.resizeHandles = "e, w";
+    }
+
+    getGridOptions() {
+        return {
+            // Trigger overflow-y:scroll after 650px height
+            sizeToContent: 650
+        };
+    }
+
+    getMarkup() {
+        let $container = $('<div></div>');
+        let $caddyDomainTable = this.createTable('caddyDomainTable', {
+            headerPosition: 'none'
+        });
+
+        $container.append($caddyDomainTable);
+        return $container;
+    }
+
+    async onWidgetTick() {
+        try {
+            // Check if caddy is enabled
+            const data = await ajaxGet('/api/caddy/reverse_proxy/get', {});
+            if (!data.caddy.general || data.caddy.general.enabled === "0") {
+                this.displayError(`${this.translations.unconfigured}`);
+                return;
+            }
+
+            // Process domains if caddy is enabled
+            let domains = { ...data.caddy.reverseproxy.reverse, ...data.caddy.reverseproxy.subdomain };
+            this.processDomains(domains);
+
+        } catch (error) {
+            this.displayError(`${this.translations.error}`);
+        }
+    }
+
+    // Utility function to display errors within the widget
+    displayError(message) {
+        const $error = $(`<div class="error-message"><a href="/ui/caddy/general">${message}</a></div>`);
+        $('#caddyDomainTable').empty().append($error);
+    }
+
+    processDomains(domains) {
+        if (!this.dataChanged('domains', domains)) {
+            return;
+        }
+
+        let rows = [];
+        // Assuming domains is a combination of both reverse and subdomains
+        for (const key in domains) {
+            const domain = domains[key];
+            let colorClass = domain.enabled === "1" ? 'text-success' : 'text-danger';
+            let tooltipText = domain.enabled === "1" ? this.translations.enabled : this.translations.disabled;
+            let domainPort = domain.FromDomain;
+
+            if (domain.FromPort) {
+                domainPort += `:${domain.FromPort}`;
+            }
+
+            let row = $(`
+                <div class="caddy-info">
+                    <div class="caddy-enabled">
+                        <i class="fa fa-globe ${colorClass}" style="cursor: pointer;"
+                            data-toggle="tooltip" title="${tooltipText}">
+                        </i>
+                        &nbsp;
+                        <a class="caddy-domainport" href="/ui/caddy/reverse_proxy">
+                            ${domainPort}
+                        </a>
+                    </div>
+                </div>
+            `).prop('outerHTML');
+
+            rows.push({ html: row, enabled: domain.enabled });
+        }
+
+        // Sort rows by their enabled status
+        rows.sort((a, b) => a.enabled - b.enabled);
+
+        // Update table with sorted rows
+        super.updateTable('caddyDomainTable', rows.map(row => [row.html]));
+
+        // Initialize tooltips for interactivity
+        $('[data-toggle="tooltip"]').tooltip();
+    }
+}

www/caddy/src/opnsense/www/js/widgets/Metadata/Caddy.xml

@@ -0,0 +1,29 @@
+<metadata>
+    <caddydomain>
+        <filename>CaddyDomain.js</filename>
+        <endpoints>
+            <endpoint>/api/caddy/reverse_proxy/*</endpoint>
+        </endpoints>
+        <translations>
+            <title>Caddy Domains</title>
+            <enabled>Enabled</enabled>
+            <disabled>Disabled</disabled>
+            <unconfigured>Caddy is disabled or not configured.</unconfigured>
+        </translations>
+    </caddydomain>
+    <caddycertificate>
+        <filename>CaddyCertificate.js</filename>
+        <endpoints>
+            <endpoint>/api/caddy/*</endpoint>
+        </endpoints>
+        <translations>
+            <title>Caddy Certificates</title>
+            <valid>Valid</valid>
+            <expired>Expired</expired>
+            <unconfigured>Caddy is disabled or not configured.</unconfigured>
+            <nocerts>Caddy does not manage any automatic certificates.</nocerts>
+            <expires>Expires:</expires>
+            <days>days</days>
+        </translations>
+    </caddycertificate>
+</metadata>