octoprint on foosel.net

How to quickly create a header modifying reverse proxy with mitmproxy

Tue, 12 Mar 2024 00:00:00 +0000

I’m currently in the process of testing some changes on OctoPrint involving its automatic user login via request headers, and for that needed to quickly set up a reverse proxy that would modify the headers of the requests going to the development server for some quick testing.

Specifically, I wanted a quick CLI tool that would allow me to set up a reverse proxy listening on port 5555, forwarding to http://localhost:5000 while also setting the headers X-Remote-User to remote and X-Remote-Host to localhost:5555.

Enter mitmproxy, or more specifically its mitmdump tool, which turned out to be a great tool for this job.

All I needed was to run the following command:

mitmdump --mode reverse:http://localhost:5000@5555 --modify-headers "/X-Remote-User/remote" --modify-headers "/X-Forwarded-Host/localhost:5555"

This does the following:

--mode reverse:http://localhost:5000@5555 sets up a reverse proxy listening on port 5555, forwarding to http://localhost:5000
--modify-headers "/X-Remote-User/remote" sets the X-Remote-User header to remote
--modify-headers "/X-Forwarded-Host/localhost:5555" sets the X-Forwarded-Host header to localhost:5555

With that the reverse proxy test page in OctoPrint turned all green and I could test my changes without having to set up an actual reverse proxy in front of the development server.

How to use Obsidian's Dataview plugin to visualize frontmatter

Fri, 14 Apr 2023 00:00:00 +0000

For every OctoPrint release I run through several update tests: I flash a specific OctoPi version, push it to a specific OctoPrint version, configure the release channel, then see if updating to the newest release (candidate) works fine. I use my testrig and its automation scripts for that and usually go through something between 5 and 10 separate scenarios.

So far all of these scenarios were noted down as a Markdown table in my release checklist that these days I prepare in my Obsidian vault, including manually adjusting the testrig commands to match the scenario. Having to take care of the latter is something that has been annoying for a long time now, and during the preparation for yesterday’s release candidate I decided enough is enough and looked into improving my tooling a bit. In the end, I used Obsidian’s quite amazing Dataview plugin to query the information about the planned test scenarios from the checklist’s frontmatter, build the testrig command from that, then render all of this as a table, complete with some checkboxes for state logging during the tests and a copy button for the command.

Here’s the format of the frontmatter for the latest release candidate 1.9.0rc5:

update_test_type: maintenance
update_tests:
- image: 0.17.0-py3
  update_test_type: simplepip
  additional: pip=21.3.1
  wip: true
  done: true
- image: 0.18.0
  channel: maintenance
  start: 1.8.7
  wip: true
  done: true
- image: 0.18.0-latest
  channel: maintenance
  wip: true
  done: true
- image: 1.0.0
  channel: maintenance
  start: 1.8.7
  wip: true
  done: true
- image: 1.0.0-latest
  channel: stable
  wip: true
  done: true
- image: 1.0.0-latest
  channel: maintenance
  start: 1.9.0rc3
  wip: true
  done: true
- image: 1.0.0-latest
  channel: maintenance
  start: 1.9.0rc4
  wip: true
  done: true

And this is the dataviewjs query I used:

const createCheckbox = (checked) => {
	const checkbox = this.container.createEl("input", {"type": "checkbox"});
	checkbox.checked = checked;
	return checkbox;
}
const createDropdown = (options, selected) => {
	const dropdown = this.container.createEl("select");
	options.map((value, idx) => {
		const option = dropdown.createEl("option", {"text": value, "value": value});
		option.selected = (value == selected);
	});
	return dropdown;
}
const createCopyButton = (command) => {
	const button = this.container.createEl("button", {"text": "Copy"})
	button.addEventListener("click", (evt) => {
		evt.preventDefault();
		navigator.clipboard.writeText(command);
	});
	return button;
}

const DUTS = ["pia", "pib", "pic"];

const update_type = dv.current().update_test_type || "maintenance";
const tests = dv.current().update_tests;

dv.table(["ID", "WIP", "Done", "DUT", "OctoPi", "Channel", "Start", "Command", ""], tests
	.map((t, idx) => {
		const dut = t.dut || DUTS[idx % DUTS.length];
		const version = (!t.start || t.start == "stock") ? "" : `,version=${t.start}`;

		let command;
        switch (t.update_test_type) {
			case "simplepip": {
				command = `fab flashhost_flash_and_provision:${t.image} octopi_test_simplepip`;
				if (t.additional) {
					command += ":" + t.additional;
				}
				break;
			}
			default: {
				command = `fab flashhost_flash_and_provision:${t.image} octopi_test_update_${update_type}:${t.channel}${version}`;
				if (t.additional) {
				  command += "," + t.additional
				}
				break;
			}
		}

		const wipCheckbox = createCheckbox(t.wip);
		const doneCheckbox = createCheckbox(t.done);
		const dutDropdown = createDropdown(DUTS, dut);
		const copyButton = createCopyButton(command);

		return [idx + 1, wipCheckbox, doneCheckbox, dutDropdown, t.image, t.channel, t.start ? t.start : "stock", "`" + command + "`", copyButton]
	}));

This iterates over the list of scenarios, and for each generates a command, assigns one of the DUTs (Device Under Test) of the testrig, creates checkboxes for WIP and Done seeded from the frontmatter data and also a copy button. This then gets rendered as a table.

Throwing that query in a dataviewjs typed markdown code fence yields a nice rendition of the data that allows me to check if I have forgotten anything, track my progress with the checkboxes and also allows me to easily copy the generated command with the click of a button, so I don’t have to manually copy paste stuff anymore:

What I have not yet figured out is how to manipulate the frontmatter directly via the checkboxes and DUT dropdowns (for logging purposes) - right now I still have to edit the state changes manually to persist them. The MetaEdit plugin for Obsidian looked promising, but that doesn’t seem to be flexible enough to manipulate a list of items (yet).

In any case, this helped me a ton yesterday, and will save me a lot of time with future releases and release candidates!

How to grep a log for multiline errors

Wed, 01 Feb 2023 00:00:00 +0000

I just found myself in the position to have to grep an OctoPrint log file for error log entries with attached Python stack traces. I wanted to not only get the starting line where the exception log output starts, but the full stack trace up until the next regular log line.

The format of the lines in octoprint.log is a simple %(asctime)s - %(name)s - %(levelname)s - %(message)s, so a log with an error and attached exception looks like this:

2023-01-30 17:50:45,704 - octoprint.events.fire - DEBUG - Firing event: Disconnecting (Payload: None)
2023-01-30 17:50:45,704 - octoprint.events - DEBUG - Sending action to >
2023-01-30 17:50:45,705 - octoprint.plugin - DEBUG - Calling on_event on action_command_notification
2023-01-30 17:50:45,705 - octoprint.plugin - DEBUG - Calling on_event on action_command_prompt
2023-01-30 17:50:45,705 - octoprint.plugin - DEBUG - Calling on_event on announcements
2023-01-30 17:50:45,705 - octoprint.plugin - DEBUG - Calling on_event on file_check
2023-01-30 17:50:45,706 - octoprint.plugin - DEBUG - Calling on_event on firmware_check
2023-01-30 17:50:45,706 - octoprint.plugin - DEBUG - Calling on_event on pluginmanager
2023-01-30 17:50:45,706 - octoprint.plugin - DEBUG - Calling on_event on softwareupdate
2023-01-30 17:50:45,706 - octoprint.plugin - DEBUG - Calling on_event on tracking
2023-01-30 17:50:45,711 - octoprint.plugin - DEBUG - Calling on_event on mqtt
2023-01-30 17:50:45,732 - octoprint.events.fire - DEBUG - Firing event: Disconnected (Payload: None)
2023-01-30 17:50:45,735 - octoprint.events - DEBUG - Sending action to . at 0x000001635BDB2CA0>
2023-01-30 17:50:45,750 - octoprint.events - ERROR - Got an exception while sending event Disconnected (Payload: None) to . at 0x000001635BDB2CA0>
Traceback (most recent call last):
  File "C:\Devel\OctoPrint\OctoPrint\src\octoprint\events.py", line 197, in _work
    listener(event, payload)
  File "C:\Devel\OctoPrint\OctoPrint\src\octoprint\server\__init__.py", line 1212, in 
    octoprint.events.Events.DISCONNECTED, lambda e, p: run_autorefresh()
                                                       ^^^^^^^^^^^^^^^^^
  File "C:\Devel\OctoPrint\OctoPrint\src\octoprint\server\__init__.py", line 1195, in run_autorefresh
    autorefresh.stop()
    ^^^^^^^^^^^^^^^^
AttributeError: 'RepeatedTimer' object has no attribute 'stop'
2023-01-30 17:50:45,753 - octoprint.events - DEBUG - Sending action to >
2023-01-30 17:50:45,755 - octoprint.plugin - DEBUG - Calling on_event on action_command_notification
2023-01-30 17:50:45,756 - octoprint.server.util.sockjs - DEBUG - Socket message held back until permissions cleared, added to backlog: plugin
2023-01-30 17:50:45,758 - octoprint.plugins.action_command_notification - INFO - Notifications cleared
2023-01-30 17:50:45,758 - octoprint.plugin - DEBUG - Calling on_event on action_command_prompt
2023-01-30 17:50:45,758 - octoprint.plugin - DEBUG - Calling on_event on announcements
2023-01-30 17:50:45,759 - octoprint.plugin - DEBUG - Calling on_event on file_check
2023-01-30 17:50:45,759 - octoprint.plugin - DEBUG - Calling on_event on firmware_check
2023-01-30 17:50:45,759 - octoprint.server.util.sockjs - DEBUG - Socket message held back until permissions cleared, added to backlog: plugin
2023-01-30 17:50:45,763 - octoprint.plugin - DEBUG - Calling on_event on pluginmanager
2023-01-30 17:50:45,764 - octoprint.plugin - DEBUG - Calling on_event on softwareupdate
2023-01-30 17:50:45,764 - octoprint.plugin - DEBUG - Calling on_event on tracking

What I now wanted is for grep to spit out just the ERROR line and the attached stack trace:

2023-01-30 17:50:45,750 - octoprint.events - ERROR - Got an exception while sending event Disconnected (Payload: None) to . at 0x000001635BDB2CA0>
Traceback (most recent call last):
  File "C:\Devel\OctoPrint\OctoPrint\src\octoprint\events.py", line 197, in _work
    listener(event, payload)
  File "C:\Devel\OctoPrint\OctoPrint\src\octoprint\server\__init__.py", line 1212, in 
    octoprint.events.Events.DISCONNECTED, lambda e, p: run_autorefresh()
                                                       ^^^^^^^^^^^^^^^^^
  File "C:\Devel\OctoPrint\OctoPrint\src\octoprint\server\__init__.py", line 1195, in run_autorefresh
    autorefresh.stop()
    ^^^^^^^^^^^^^^^^
AttributeError: 'RepeatedTimer' object has no attribute 'stop'

For this I needed a way to set grep to match multiple lines and do a (non-matching) look ahead for the end. It turns out that the secret to success here is to treat the whole input as one line, use Perl compatible regex mode, and make sure to set the multiline flag. After some fiddling around on regex101.com and reading up on Perl’s regex options¹, I came up with the following:

grep -Pazo '(?m)^\N+\- ERROR \-\N*\n(^\N*?\n)*?(?=\d{4}-\d{2}-\d{2} \d{2}:\d{2}:\d{2},\d{3} \- )' octoprint.log

Let’s walk through this:

-P enables Perl compatible regex mode
-a enables text mode
-z turns all newlines into null bytes and thus treats the whole input as a single line for finding matches²
-o only outputs the matched part of the line (otherwise we’d get the whole file printed out)
(?m) enables multiline mode
^\N+\- ERROR \-\N*\n matches the first line of the error, which is the one that starts with the timestamp and package and contains the word ERROR
(^\N*?\n)*? non-greedily matches all following lines of the error, which are anything but a newline followed by a newline
(?=\d{4}-\d{2}-\d{2} \d{2}:\d{2}:\d{2},\d{3} \- ) is a positive look-ahead that matches a line starting with a timestamp again, which signifies the end of the error’s lines

Hooray, it works 🥳:

❯ grep -Pazo '(?m)^\N+\- ERROR \-\N*\n(^\N*?\n)*?(?=\d{4}-\d{2}-\d{2} \d{2}:\d{2}:\d{2},\d{3} \- )' octoprint.log
2023-01-30 17:50:45,750 - octoprint.events - ERROR - Got an exception while sending event Disconnected (Payload: None) to . at 0x000001635BDB2CA0>
Traceback (most recent call last):
  File "C:\Devel\OctoPrint\OctoPrint\src\octoprint\events.py", line 197, in _work
    listener(event, payload)
  File "C:\Devel\OctoPrint\OctoPrint\src\octoprint\server\__init__.py", line 1212, in 
    octoprint.events.Events.DISCONNECTED, lambda e, p: run_autorefresh()
                                                       ^^^^^^^^^^^^^^^^^
  File "C:\Devel\OctoPrint\OctoPrint\src\octoprint\server\__init__.py", line 1195, in run_autorefresh
    autorefresh.stop()
    ^^^^^^^^^^^^^^^^
AttributeError: 'RepeatedTimer' object has no attribute 'stop'

(And yes, I’ve fixed the error that lead to this stack trace as well 😉)

I don’t know about you, but I always forget about positive/negative look-ahead/behind and pattern-match modifiers. ↩︎
The downside of this is that now -n (print line number of match) will not work anymore and just happily report line 1 for every single match. ↩︎