[Blink] Fwd: Let's Encrypt DST Root CA X3 Expiration

[Jeff Pyle]

Complex indeed.


> The idea as I understand it is that browsers should
> see the ISRG Root X1 certificate, realize that it itself is already
> trusted by the OS or browser, and not even look at the next expired
> DST Root CA X3 certificate in the chain.
>

This doesn't work sometimes on some clients.  I ran into a problem
yesterday where a batch of Polycom VVX phones would no longer register SIP
via TLS or pull provisioning via HTTPS, and the root cause (pun intended)
was this issue.  Loading the ISRG Root X1 certificate alone didn't solve
the problem; the phone still complained the server's cert was expired
during the SSL handshake.  To move past it, I had to change the TLS profile
on the Polycom to use only the manually loaded CA cert (Platform CA 1) and
ignore anything built-in.  And then it was fine.

I found this site illustrative:  https://chainchecker.certifytheweb.com/
Pointing it at ag-projects.com, or any system using Let's Encrypt, yields
the same results:

ag-projects.com ▶▶ R3 ▶▶ ISRG Root X1 ▶▶ [DST Root CA X3]
Let's Encrypt Legacy Chain (Supports older devices)

This Let's Encrypt chain includes the expired DST Root CA X3 in order
to remain compatible with older operating system such as Android 7.0
and lower.


I fear the [DST Root CA X3] cert listed there, the expired one, is going to
do more harm than good on many devices, like it did on my Polycom VVXs.  It
wouldn't surprise me if Let's Encrypt changes their policy in such a way
that no longer includes it.


- Jeff


On Tue, Oct 5, 2021 at 7:16 PM Adrian Georgescu <ag at ag-projects.com> wrote:

> The complexity of the problem…
>
> Begin forwarded message:
>
> *From: *raf <macports at raf.org>
> *Subject: **Re: Let's Encrypt DST Root CA X3 Expiration*
> *Date: *2 October 2021 at 23:32:45 GMT-3
> *To: *macports-users at lists.macports.org
>
> On Sat, Oct 02, 2021 at 04:14:05AM -0500, Ryan Schmidt <
> ryandesign at macports.org> wrote:
>
> macports.org and other secure web sites that use Let's Encrypt may
> no longer be accessible to you if you use older versions of macOS
> or older browsers or user agents. For example, the libcurl in macOS
> 10.14 can't talk to many Let's Encrypt web sites now, including
> distfiles.macports.org and packages.macports.org, and MacPorts uses
> macOS libcurl to download things. Safari and many browsers don't use
> libcurl so they may be affected differently.
>
> Let's Encrypt is a free certificate provider used by macports.org
> and many other web sites to provide https encryption. Certificates
> they issue depend on their "ISRG Root X1" certificate which was
> cross-signed by the "DST Root CA X3" certificate, because DST Root
> CA X3 was more widely accepted by browsers when Let's Encrypt got
> started years ago. Both of these root certificates are included in the
> certificate chain served by web sites that use Let's Encrypt.
>
> ISRG Root X1 itself has been trusted by browsers for some time
> now and DST Root CA X3 expired a couple days ago on September 30,
> 2021. Apparently in order to provide the widest compatibility,
> certificate chains continue to list the old expired root certificate
> after the new one. The idea as I understand it is that browsers should
> see the ISRG Root X1 certificate, realize that it itself is already
> trusted by the OS or browser, and not even look at the next expired
> DST Root CA X3 certificate in the chain.
>
> They advertised this root certificate expiration as being a very minor
> situation, but unfortunately it seems that a large portion of Apple
> devices cannot deal with this change. On many Macs, it seems that the
> entire certificate chain is being validated, and the expired extra
> root certificate is causing the connection to be disallowed. What
> alerted me to the problem in the first place was seeing many failed
> builds on our Buildbot system due to fetch failures.
>
> I'm not certain what to do to address this. On the web servers
> we control, we can apparently remove the expired DST Root CA X3
> certificate from the chain that we send. That will help those
> systems that already trust ISRG Root X1. I'm not sure how far back
> that is. For older systems, we can modify master_sites.tcl and
> archive_sites.tcl to change which OS versions try to access our mirror
> servers via https. None of this necessarily helps our build server be
> able to mirror distfiles in the first place. If you have ideas, let me
> know.
>
> https://letsencrypt.org/docs/dst-root-ca-x3-expiration-september-2021/
>
>
> There is a discussion on the LetsEncrypt community site
> with instructions for installing the ISRG Root X1
> certificate on older versions of macOS:
>
>
> https://community.letsencrypt.org/t/connection-errors-on-apple-devices/161107/16
>
> Here are instructions for 10.10 and 10.11:
>
>
> https://community.letsencrypt.org/t/connection-errors-on-apple-devices/161107/25
>
> Here's another approach that worked on 10.9.5 and
> 10.11.6 (i.e., override the expiry by always trusting
> DST Root CA X3:
>
>
> https://community.letsencrypt.org/t/connection-errors-on-apple-devices/161107/28
>
> Here's my approach for 10.6.8 because the above didn't
> work (i.e., export root certificates from a later macOS
> and import them):
>
>
> https://community.letsencrypt.org/t/connection-errors-on-apple-devices/161107/36
>
> But these are all client-side fixes. The only
> server-side fix seems to be to change to a different
> certificate authority. I didn't see any mention of
> removing the DST Root CA X3 certificate from the chain.
> That would probably only work from macOS 10.12 onwards.
> ISRG Root X1 is only trusted by macOS since 10.12.
> Earlier than that, it needs to be added.
>
> The rest is a bit rambly. It might be best to just
> consult the LetsEncrypt community forum above.
>
> I should mention that I didn't notice any problems with
> macports on 10.6.8. curl has its own root certificates
> and they are fine. And I was able to do an upgrade. But
> I might have already installed the ISRG Root X1
> certificate at least into my "local" keychain before
> trying it (or maybe into the "System" and "System
> Roots" keychains).
>
> However, I still don't think that it's entirely OK.
> Firefox on 10.6.8 can access macports.org with no
> problem (but it certainly wasn't accessing my
> LetsEncrypt-certified sites beforehand), but Sarafi
> tells me that it can't verify macports.org's identity,
> but it still lets me access it. If I quit and restart
> Safari, and visit macports.org, it does the same thing.
> Firefox uses its own root certificates. Safari uses the
> system ones. But I definitely have ISRG Root X1 in both
> the "System" and "System Roots" keychains. So I don't
> know why Safari has a problem. In Keychain Access, it's
> marked as "Always Trust" but it also says "trusted for
> macports.org" (I don't know why it says that). That's
> confusing. But in Safari, when viewing the certificate,
> there was a checkbox labelled "Always trust". After
> checking it, quitting and restarting Safari, it visited
> macports.org without complaint.
>
> I should also mention that I can't find DST Root CA X3
> in my 10.6.8 keychains. So that's wierd. Otherwise, I
> probably should have set it to always trust (not that
> having ISRG Root X1 set to always trust convinced
> Safari to trust it immediately - I had to tell Safari
> itself to trust it as well).
>
> cheers,
> raf
>
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.ag-projects.com/pipermail/blink/attachments/20211005/82272e42/attachment.htm>