Solved Can't download crates (forbidden) while trying to port rust application

[mariachi]

TL;DR: make sure you have the latest version of the ports tree, this commit fixes the issue by switching to the crates.io CDN https://codeberg.org/FreeBSD/freebsd-ports/commit/bc18f00c490cd54dfe38e2d94d8c4feae2815f0f. I'm keeping the full story here as reference to what I had to do to figure it out

I had a working port for an earlier version of chhoto-url but now I can't get past make makesum after trying to update the Makefile to the latest release.
I'm not using poudriere.

> ls
Makefile                Makefile.crates         distinfo                files/                  pkg-descr               pkg-plist
> doas make makesum
/!\ chhoto-url-7.7.0: Makefile warnings, please consider fixing /!\

Not validating first entry in CATEGORIES due to being outside of PORTSDIR.
Please ensure this is proper when committing.

===>  License MIT accepted by the user
/!\ chhoto-url-7.7.0: Makefile warnings, please consider fixing /!\

Not validating first entry in CATEGORIES due to being outside of PORTSDIR.
Please ensure this is proper when committing.

===>  License MIT accepted by the user
===>   chhoto-url-7.7.0 depends on file: /usr/local/sbin/pkg - found
===> Fetching all distfiles required by chhoto-url-7.7.0 for building
> make cargo-crates > Makefile.crates
> doas make makesum
/!\ chhoto-url-7.7.0: Makefile warnings, please consider fixing /!\

Not validating first entry in CATEGORIES due to being outside of PORTSDIR.
Please ensure this is proper when committing.

===>  License MIT accepted by the user
/!\ chhoto-url-7.7.0: Makefile warnings, please consider fixing /!\

Not validating first entry in CATEGORIES due to being outside of PORTSDIR.
Please ensure this is proper when committing.

===>  License MIT accepted by the user
===>   chhoto-url-7.7.0 depends on file: /usr/local/sbin/pkg - found
=> rust/crates/actix-codec-0.5.4.crate doesn't seem to exist in /usr/ports/distfiles/chhoto-url.
=> Attempting to fetch https://crates.io/api/v1/crates/actix-codec/0.5.4/download?dummy=/rust/crates/actix-codec-0.5.4.crate
fetch: https://crates.io/api/v1/crates/actix-codec/0.5.4/download?dummy=/rust/crates/actix-codec-0.5.4.crate: Forbidden
=> Attempting to fetch http://distcache.FreeBSD.org/ports-distfiles/chhoto-url/rust/crates/actix-codec-0.5.4.crate
fetch: http://distcache.FreeBSD.org/ports-distfiles/chhoto-url/rust/crates/actix-codec-0.5.4.crate: Not Found
=> Couldn't fetch it - please try to retrieve this
=> port manually into /usr/ports/distfiles/chhoto-url and try again.
*** Error code 1

Stop.
make[1]: stopped making "fetch" in /home/santi/code/freebsd-ports/www/chhoto-url
*** Error code 1

Stop.
make: stopped making "makesum" in /home/santi/code/freebsd-ports/www/chhoto-url

running
fetch https://crates.io/api/v1/crates/actix-codec/0.5.4/download?dummy=/rust/crates/actix-codec-0.5.4.crate by hand fails with Forbidden too, however, using curl instead succeeds.

Here's my current Makefile:

PORTNAME=       chhoto-url
DISTVERSION=    7.7.0
CATEGORIES=     www
DISTNAME=       ${DISTVERSION}
DIST_SUBDIR=    ${PORTNAME}

MAINTAINER=     me@jcpsantiago.xyz
COMMENT=        URL shortening web application written in Rust
WWW=            https://github.com/SinTan1729/chhoto-url

LICENSE=        MIT
LICENSE_FILE=   ${WRKSRC}/LICENSE

USES=           cargo
USE_RC_SUBR=    chhoto_url

USE_GITHUB=     yes
GH_ACCOUNT=     SinTan1729

CARGO_BUILD_ARGS=       --target-dir=backend/target
CARGO_INSTALL_PATH=     backend
CARGO_CARGOTOML=        ${WRKSRC}/backend/Cargo.toml
CARGO_CARGOLOCK=        ${WRKSRC}/backend/Cargo.lock

LIB_DEPENDS+=libzstd.so:archivers/zstd

USERS=          www
GROUPS=         www

SUB_FILES=      chhoto_url.conf.sample chhoto_url
SUB_LIST=       WWWDIR=${WWWDIR}

PLIST_SUB=      WWWDIR_REL=${WWWDIR_REL} \
                ETCDIR_REL=${ETCDIR_REL}

post-install:
        # NOTE: cargo will produces the binary `chhoto-url`, but
        # rc doesn't like dashes, so we rename the binary to match the rc
        ${MV} ${STAGEDIR}${PREFIX}/bin/chhoto-url ${STAGEDIR}${PREFIX}/bin/chhoto_url

        (cd ${WRKSRC}/resources && ${COPYTREE_SHARE} . ${STAGEDIR}${WWWDIR}/resources)

        # NOTE: chhoto-url will explode if the database file does not exist
        # when the app starts. This ensures the file is there beforehand.
        ${INSTALL_DATA} /dev/null ${STAGEDIR}${WWWDIR}/urls.sqlite

        ${MKDIR} ${STAGEDIR}${ETCDIR}
        ${INSTALL_DATA} ${WRKDIR}/chhoto_url.conf.sample ${STAGEDIR}${ETCDIR}

        ${MKDIR} ${STAGEDIR}/var/log/chhoto_url
        ${MKDIR} ${STAGEDIR}/var/run/chhoto_url

.include <bsd.port.mk>

Any ideas? I'm happy to share more details, just don't know what is relevant at this stage since my experience with ports is limited.

EDIT: After further googling, I found some conversations complaining about a similar issue (see here (Reddit); unrelated to FreeBSD) and mentioning stricter user-agent checks by crates.io. Testing the same fetch command as above by hand by adding --user-agent="freebsd-fetch" succeeded; fetching without a user-agent from static.crates.io/crates instead of the crates.io/api works.
If fetch must use a user-agent or point at a different repo (static.crates.io in this case), shouldn't all rust ports be failing at this point? Otherwise, what did I miss regarding configuring fetch? I can't find a specific user-agent mention in the handbook.

I also found this but clicking it takes me to the "Most recent commits" page which loads a day at a time and I have no idea when this was written.

so I checked the source code and found the commit where this was changed https://codeberg.org/FreeBSD/freebsd-ports/commit/bc18f00c490cd54dfe38e2d94d8c4feae2815f0f

I deleted the ports tree I had since I installed FreeBSD and pulled the latest main branch instead and now everything works.
I've updated the top of this post.