Commit graph

16056 commits

Author SHA1 Message Date
Oneric
6e7dee552a federation: let http_signatures library handle request aliases
This avoids spurious key refetches on each failing alias
2025-06-07 20:27:58 +02:00
Oneric
8dad70e8e7 instances: drop has_request_signatures
This property was introduced as a way to gauge whether and
how much enabling authfetch might break passive federation in
https://akkoma.dev/AkkomaGang/akkoma/pulls/312.

However, with the db field defaulting to false, there’s no distinction
between instances without valid signatures and those which just never
attempted to fetch anything from the local instance.
Furthermore, this was never exposed anywhere and required manually
checking the database or cachex state via a remote shell.

Given the above it appears this doesn't actually
provide anything useful, thus drop it.
2025-06-07 20:27:58 +02:00
Oneric
f2ca71f1ad Adapt to new http_signature API 2025-06-07 20:27:58 +02:00
Oneric
fefc884f22 Drop EnsureUserPublicKey plug
It is not needed since fetch_public_keys will already
initiate remote lookup if necessary
2025-06-07 20:27:58 +02:00
Oneric
d53a779c35 Merge pull request 'NormalizeMarkup scrub contentMap' (#932) from Oneric/akkoma:normalize-markup_contentmap into develop
Reviewed-on: https://akkoma.dev/AkkomaGang/akkoma/pulls/932
2025-06-07 18:27:26 +00:00
Oneric
1b7d9a0f76 Merge pull request 'More federation backoff tweaks' (#934) from Oneric/akkoma:federation_backoff_tweaks into develop
Reviewed-on: https://akkoma.dev/AkkomaGang/akkoma/pulls/934
2025-06-07 18:24:26 +00:00
Oneric
258841c310 federation/in: space out receiver retries more
The most common permanent receiver error arises for likes/boosts
when we don’t yet know the rlevant object and can't fetch it
due to the remote being overwhelmed or otherwise down.

Before this changes all retries were rather rapid
thus not giving the remote enough time to recover
and usually all failing. Now the remote has about 20
minutes to recover before we give up.

Transient errors from race conditions and (presumably)
weird database-cache interactions also occur regularly.
However, they resolve within the first one or two retries
and those intial retries still happen relatively quickly.
2025-05-18 14:24:09 +02:00
Oneric
50afbc63e4 federation/out: raise publisher retries by one
We now tolerate a whole day of downtime by default
instead of only about three hours.
2025-05-18 14:24:08 +02:00
Oneric
782a222efd common_api: make inserted attachment links scrubber-compliant 2025-05-16 21:30:26 +02:00
Oneric
2eadc4b513 mrf/normalize_markup: also scrub contentMap
Only scrubbing "content" leads to differences between
"content" and "contentMap" eventhough the latter should
ideally match the former exactly for the primary language’s entry.

While ideally, for locally generated posts there should be no difference
between applying the scrubber or not, as it turns out automatically
generated attachment links didn't match the form expected by our default
scrubber.

Currently Akkoma never uses nor exposes the value of contentMap entries,
thus this oversight was harmless wrt to safety and at most pertubed
the language detection for our posts perfomed by remote servers.

Fixes: https://akkoma.dev/AkkomaGang/akkoma/issues/928
2025-05-16 21:30:26 +02:00
Oneric
3791b695f4 Merge pull request 'fix some elixir 1.18 deprecation warnings' (#930) from Oneric/akkoma:build_deprecation-warnings into develop
Reviewed-on: https://akkoma.dev/AkkomaGang/akkoma/pulls/930
2025-05-16 09:39:24 +00:00
Oneric
c749df62a1 cosmetic: replace deprecated Tuple.append
Everytime a tuple is append a new one is allocated and all data copied
over. Keeping it a list until all entries are converted avoids this.
2025-05-15 23:40:57 +02:00
Oneric
88a6a9d964 cosmetic: replace deprecated comment syntax in eex
The replacement <!-- --> is available since
elixir 1.14.0 which matches our minimal version.
2025-05-15 23:07:43 +02:00
Oneric
771d3c3103 Merge pull request 'api/masto/instance: use WebFinger domain for URI' (#927) from Oneric/akkoma:mastoapi_instance-uri into develop
Reviewed-on: https://akkoma.dev/AkkomaGang/akkoma/pulls/927
2025-05-15 19:45:08 +00:00
Oneric
40fef8e632 api/masto/instance: use WebFinger domain for URI
Despite its name this property is not supposed to be a full URI,
but just a bare domain witout protocol. Furthermore, it’s supposed
to be the WebFinger domain used in userhandles and NOT the domain used
for API and ActivityPub objects (which every caller will already know
anyway).

Not following this caused issues for Pachli and Tusky.

Reported-by: nikclayton
2025-05-15 21:10:17 +02:00
Oneric
6f17eae3f9 Merge pull request 'api/masto/instance: add short_description field' (#926) from Oneric/akkoma:mastoapi_short-description into develop
Reviewed-on: https://akkoma.dev/AkkomaGang/akkoma/pulls/926
2025-05-15 19:10:02 +00:00
Oneric
295e4a4da3 api/masto/instance: add short_description field
Added in Mastodon 2.9.2 (June 2019) this is plain-text-only and supposed
to be shorter compared to the older description field.
Some clients were reported to require this field to properly function.

Reported-by: https://akkoma.dev/paulyd
2025-05-15 20:41:55 +02:00
floatingghost
5e091f44ff Merge pull request 'Properly register digest_emails queue' (#925) from Oneric/akkoma:fix_digest into develop
Reviewed-on: https://akkoma.dev/AkkomaGang/akkoma/pulls/925
2025-05-12 12:52:40 +00:00
Oneric
d837a390d9 Properly register digest_emails queue
The queue was not actually registered leading to
jobs getting scheduled each week but never processed
forever lingering in 'available' state.

Fixes omission in: a4f834a687
2025-05-11 23:07:29 +02:00
Oneric
30b1684e28 Merge pull request 'docs: fix spelling and grammar' (#923) from Oneric/akkoma:solidsanek_doc-tweaks into develop
Port of: https://akkoma.dev/AkkomaGang/akkoma/pulls/525
2025-05-10 15:28:30 +00:00
solidsanek
23ee94c0d8 docs: use highlight blocks where appropriate 2025-05-10 17:14:36 +02:00
solidsanek
303f0931de docs: fix spelling and grammar 2025-05-10 17:00:55 +02:00
solidsanek
8b05f8bd84 docs: consistenly capitalise Caddy
When not referring to the command
2025-05-10 15:15:54 +02:00
solidsanek
4766e4bfce docs: consistently cpaitlaise Apache 2025-05-10 15:14:23 +02:00
solidsanek
11e82ea508 docs: change FE names to be more consistent
Preferring the spelling used in their repository
2025-05-10 15:10:57 +02:00
Oneric
3c3134f87c docs: exactly match spelling of distro package names
And use the project’s preferred spelling when
not referring to specific distro package.

Spelling of RedHat packages was left as is
as I can’t access RedHat’s package repository.
2025-05-10 15:01:09 +02:00
solidsanek
d86ae26092 docs: consistently spell ejabberd using lowercase
This matches the project’s own spelling
2025-05-10 14:50:54 +02:00
solidsanek
873f57690a docs: consistenly spell nginx using lowercase
This matches the spelling used by the nginx project
(which differs from the spelling used by the NGINX corporation)
2025-05-10 14:49:38 +02:00
Oneric
6e0b6f2915 changelog: add missing entries since 3.15.2 2025-05-09 23:43:08 +02:00
Oneric
76bfc41d78 changelog: fix header levels 2025-05-09 23:42:15 +02:00
Oneric
7e5a5db63d Merge pull request 'dbprune: add --keep-followed and don't prune pinned posts by default' (#897) from Oneric/akkoma:mix-prune_newopts into develop
Reviewed-on: https://akkoma.dev/AkkomaGang/akkoma/pulls/897
2025-05-09 21:31:36 +00:00
Oneric
516827c356 mix/database: add keep-followed option for object pruning
This allows to retain posts and boosts of remote actors with local
follows regardless of age.
With the "full" setting this can be taken further treating such
followed actors just like local users even keeping all posts they
liked or reacated to.
2025-05-09 23:02:25 +02:00
Oneric
8576ec42ec mix/database: retain pinned objects by default
Pinned objects and their threads will be refetched
on user refresh which by default happens after a day
once a user is encountered again in any form including a mention.

We observed pruning pinned objects usually results in heavy load for
hours after a database prune due to a clogged up remote fetch queue as
pinned posts and their threads of many (most?) users get refetched.

Thus do not prune pinned posts by default.
Keeping closer to earlier behaviour this will still prune threads of
pinned posts regardless of --keep-threads if nothing else prevenets it.
2025-05-09 23:00:14 +02:00
Oneric
be5312228f cosmetic/mix/database: refactor log_message building
This is more concise and readable
2025-05-09 22:57:01 +02:00
Oneric
9846460702 cosmetic/mix/database: refactor object prune into subfunctions
Statmenets for keeping and breaking threads vastly differ
and the whole if block doesn't even fit on one screen.
Thus move each version out into its own function to
improve readability
2025-05-09 22:57:01 +02:00
Oneric
6c3ea04078 Merge pull request 'worker/receiver: don’t retry unsupported actions' (#902) from Oneric/akkoma:receiver_ignore_unsupported_actions into develop
Reviewed-on: https://akkoma.dev/AkkomaGang/akkoma/pulls/902
2025-05-09 20:56:19 +00:00
Oneric
f576807f1b worker/receiver: don't retry unsupported actions
Observed for e.g. user delete Undos and Bite activities
2025-05-09 22:29:49 +02:00
Oneric
487473cd75 Merge pull request 'web/metadata: provide alternate link for ActivityPub' (#905) from Oneric/akkoma:metadata_aplink into develop
Reviewed-on: https://akkoma.dev/AkkomaGang/akkoma/pulls/905
2025-05-09 20:20:04 +00:00
Oneric
c3d163d34d Merge pull request 'mediaproxy: proxy network-path references' (#903) from Oneric/akkoma:mediaproxy_networkpathref into develop
Reviewed-on: https://akkoma.dev/AkkomaGang/akkoma/pulls/903
2025-05-09 20:13:55 +00:00
Oneric
8cdfbf872d Merge pull request 'federation/out: tweak publish retry backoff' (#884) from Oneric/akkoma:publish_backoff into develop
Reviewed-on: https://akkoma.dev/AkkomaGang/akkoma/pulls/884
2025-05-09 20:12:56 +00:00
Oneric
13940a558a Merge pull request 'Expose stats about finally failed AP deliveries in prometheus' (#882) from Oneric/akkoma:telemetry-failed-deliveries into develop
Reviewed-on: https://akkoma.dev/AkkomaGang/akkoma/pulls/882
2025-05-09 20:12:01 +00:00
Oneric
aac5493dd5 Merge pull request 'Don’t pretend internal actors have follow(er|ing) collections' (#856) from Oneric/akkoma:fetch-actor-follow-collections into develop
Reviewed-on: https://akkoma.dev/AkkomaGang/akkoma/pulls/856
2025-05-09 20:10:41 +00:00
Oneric
d6f5f4db18 Merge pull request 'receiver_worker: prevent duplicate jobs' (#886) from Oneric/akkoma:receive_dedupe into develop
Reviewed-on: https://akkoma.dev/AkkomaGang/akkoma/pulls/886
2025-05-09 19:13:14 +00:00
floatingghost
fffac8bc7c Merge pull request 'Always prefer shared inbox' (#881) from Oneric/akkoma:shared-inbox into develop
Reviewed-on: https://akkoma.dev/AkkomaGang/akkoma/pulls/881
2025-05-09 16:54:19 +00:00
floatingghost
9a46685e64 Merge pull request 'Set cleanup_attachments to true by default' (#910) from norm/akkoma:cleanup-attachments-default-true into develop
Reviewed-on: https://akkoma.dev/AkkomaGang/akkoma/pulls/910
Reviewed-by: Oneric <oneric@noreply.akkoma>
2025-05-09 16:49:49 +00:00
Norm
8712e06d27 Set cleanup_attachments to true by default
Since bcfbfbcff5 (part of
<https://akkoma.dev/AkkomaGang/akkoma/pulls/789>), the overhead for
cleaning up orphaned attachments has been drattically reduced.

Most admins are unaware of this option even existing, but may notice an
increase in the size of the uploads directory (or S3 bucket size if used
instead) even if auto-expiring posts are used. This should hopefully
make this problem more manageable.

For those that still encounter performance issues, the setting can still
be disabled if needed.
2025-05-08 18:12:56 -04:00
Oneric
a80444041c federation: always prefer the shared inbox
In theory a pedantic reading of the spec indeed suggests
DMs must only be delivered to personal inboxes. However,
in practice the normative force of real-world implementations
disagrees. Mastodon, Iceshrimp.NET and GtS (the latter notably has a
config option to never use sharedInboxes) all unconditionally prefer
sharedInbox for everything without ill effect. This saves on duplicate
deliveries on the sending and processing on the receiving end.
(Typically the receiving side ends up rejecting
 all but the first copy as duplicates)

Furthermore current determine_inbox logic also actually needs up
forcing personal inboxes for follower-only posts, unless they
additionally explicitly address at least one specific actor.
This is even much wasteful and directly contradicts
the explicit intent of the spec.

There’s one part where the use of sharedInbox falls apart,
namely spec-compliant bcc and bto addressing. AP spec requires
bcc/bto fields to be stripped before delivery and then implicitly
reconstructed by the receiver based on the addressed personal inbox.
In practice however, this addressing mode is almost unused. Neither of
the three implementations brought up above supports it and while *oma
does use bcc for list addressing, it does not use it in a spec-compliant
way and even copies same-host recipients into cc before delivery.
Messages with bcc addressing are handled in another function clause,
always force personal inboxes for every recipient and not affected by
this commit.
In theory it would be beneficial to use sharedInbox there too for all
but bcc recipients. But in practice list addressing has been broken for
quite some time already and is not actually exposed in any frontend,
as discussed in https://akkoma.dev/AkkomaGang/akkoma/issues/812.
Therefore any changes here have virtually no effect anyway
and all code concerning it may just be outright removed.
2025-05-06 17:38:24 +02:00
Oneric
0d38385d6f publisher: don't mangle between string and atom
Oban jobs only can have string args and there’s no reason to insist on atoms here.

Plus this used unchecked string_to_atom
2025-05-06 17:38:18 +02:00
Oneric
6b97f085d8 Refresh mix.lock
Fixes omission in 936be4edbb
2025-05-06 17:24:58 +02:00
Oneric
818ddeca84 Merge pull request 'add instruction to make asdf work when logging into it with sudo' (#894) from shadowjonathan/akkoma:add-sudo-asdf into develop
Reviewed-on: https://akkoma.dev/AkkomaGang/akkoma/pulls/894
2025-05-05 20:45:50 +00:00