Troubleshooting HTTPS connections with mitmproxy

Programming Snapshot – mitmproxy

© Lead Image © sergwsq and robert hyrons,

© Lead Image © sergwsq and robert hyrons,

Article from Issue 224/2019

Finding the data zipping back and forth between the browser and server is not only interesting for snooping spies, but also for debugging developers. Mike Schilli gets you started with mitmproxy and shows how to customize it using Python scripts.

If things just don't work when you are developing a web application, the question immediately arises as to what data the browser and web server are actually exchanging. Tools for snooping on the network such as Wireshark (as well as proxies that sit between the client and the server) leave both requests and responses untouched, while logging them for inspection.

mitmproxy, which stands for man in the middle (MITM) proxy, is the king of the hill in this category; it makes the impossible possible by logging encrypted HTTPS requests. But first, let's look at the simplest, unencrypted case, for which my aging website that still uses plain old unencrypted HTTP is a great choice.

Figure 1 shows how the browser retrieves the requested page's HTML text, along with some images and JavaScript snippets from the server. The mitmproxy tool, which is available for download as binary from [1], sports a terminal user interface (UI), which displays a double arrow to the left of the current request, called Flow in mitmproxy parlance. When you press the Enter key, the detailed request data come up, as shown in Figure 2.


Use Express-Checkout link below to read the full article (PDF).

Buy this article as PDF

Express-Checkout as PDF
Price $2.95
(incl. VAT)

Buy Linux Magazine

Get it on Google Play

US / Canada

Get it on Google Play

UK / Australia

Related content

  • Socks 5

    Socks is a universal proxy protocol for TCP and UDP that allows internal hosts to securely pass the firewall and authenticates users. This article describes the latest version of the Socks proxy protocol and shows how to implement it.

  • Nginx

    The fast and practical Nginx web server is easy to configure and extend.

  • HTTPS Proxy

    How do you monitor the network when your client systems are connecting to secure web servers through HTTPS? We’ll show you how to keep watch using the Squid proxy server and share some inventive certificate tricks.

  • IMAP Proxies

    IMAP proxies like Perdition, Imapproxy, and Cyrus Aggregator help distribute mail to multiple IMAP servers. We examine some options for IMAP proxy in the Linux evironment.

  • Java Anonymous Proxy

    Many Websites log IP addresses and access times to identify users. If you don’t want to wind up as data in someone’s market research, the Java Anonymous Proxy will keep your surfing secret.

comments powered by Disqus

Direct Download

Read full article as PDF:

Price $2.95