Commit graph

16104 commits

Author SHA1 Message Date
Oneric
deb8689eba federation/in: fix inlined featured collections
Resolves interop issue with a (reverted but possibly returning) bridgy change
as was reported in the comments of
https://akkoma.dev/AkkomaGang/akkoma/issues/831.
This won't change anything for the problem originally reported there.

Notably we now always fetch the full collection (up to the configured
item count limit) instead of only using the first page if its link was
inlined.
2025-07-21 17:45:20 +00:00
Oneric
e2987402b3 Merge pull request 'Optimise CI runs' (#938) from Oneric/akkoma:ci-tweaks into develop
Reviewed-on: https://akkoma.dev/AkkomaGang/akkoma/pulls/938
Reviewed-by: floatingghost <hannah@coffee-and-dreams.uk>
2025-07-11 22:23:29 +00:00
Oneric
ea5a2a9f21 mix: replace deprecated warnings_as_errors option 2025-06-22 17:36:56 +02:00
Oneric
5a5d4cb86f ci: drop variables from files not using them at all 2025-06-22 17:36:55 +02:00
Oneric
c7b46e3799 ci: make pipelines conditional
If only steps are conditional the whole CI workflow
will be held up waiting until a slot is available to start them
just to then not do anything at all.

This allows us to drop "when" condition from individual steps
whenever it is now redundant with the top-level condition.
2025-06-22 17:36:55 +02:00
Oneric
25da959772 ci: merge lint and test pipeline
The lint pipeline spent ~7 minutes downloading and compiling
and only a few seconds actually checking the style.
The former is fully redundandt with what’s done during test anyway.
2025-06-22 17:36:55 +02:00
Oneric
d63292561b ci/test: remove superfluous matrix combinations
When include is used only the specified combinations
are executed and the full list above is ignored
2025-06-22 17:36:55 +02:00
Oneric
16d7d612ff Merge pull request 'misc: missing changelog entries + drop accepts_chat_messages + dedupe htmlMfm test payloads' (#943) from Oneric/akkoma:misc_chat+changelog+dedupehtmlMfmTest into develop
Reviewed-on: https://akkoma.dev/AkkomaGang/akkoma/pulls/943
2025-06-22 15:10:04 +00:00
Oneric
1c2407c129 test/federator: deduplicate html_mfm test payloads 2025-06-22 17:09:07 +02:00
Oneric
81285d06be db: drop accepts_chat_messages from users
Another leftover of chats wasting space.
All uses were purged in 0f132b802d.
2025-06-22 17:09:02 +02:00
Oneric
94207c425d changelog: add missing entries
Everything relevant since 6e0b6f2915
which wasn’t listed yet
2025-06-22 16:32:36 +02:00
Oneric
f2c2ec5e27 Merge pull request 'Add htmlMfm key when relevant' (#878) from ilja/akkoma:add_fep-c16b_discovery_mechanism_to_not_always_reparse_mfm into develop
Reviewed-on: https://akkoma.dev/AkkomaGang/akkoma/pulls/878
Reviewed-by: Oneric <oneric@noreply.akkoma>
2025-06-22 14:31:32 +00:00
ilja space
d200c7e11c Add extra tests + fix storing htmlMfm in database when federated
- This adds extra tests to be sure that scrubbing still happens.
- When doing this I notices that the htmlMfm key wasn't stored in the database when comming through the federator. This has been now been fixed too.
- We also test that values true, false or no attribute all work for incomming messages.
2025-06-21 11:32:22 +02:00
Oneric
7f6ebcb12e Merge pull request 'federation/in: improve reply on requests from blocked domains' (#940) from Oneric/akkoma:httpsig-mrf-reply into develop
Reviewed-on: https://akkoma.dev/AkkomaGang/akkoma/pulls/940
Reviewed-by: floatingghost <hannah@coffee-and-dreams.uk>
2025-06-10 19:17:19 +00:00
Oneric
0a9e7d4712 federation/in: improve reply on requests from blocked domains
Previously all such requests led to '401 Unauthorized'
whih might have triggered retries.
Now, to not leak any MRF info, we just indicate an
accept for POST requests without actually processing the object
and indiscriminately return "not found" for GET requests.

Notably this change also now causes all signed fetch requests from
blocked domains to be rejected even if authorized_fetch isn’t enabled.

Fixes: https://akkoma.dev/AkkomaGang/akkoma/issues/929
2025-06-10 20:43:33 +02:00
Oneric
42022d5f48 Merge pull request 'Allow fine-grained announce visibilities' (#941) from Oneric/akkoma:announce-visibility into develop
Reviewed-on: https://akkoma.dev/AkkomaGang/akkoma/pulls/941
Reviewed-by: floatingghost <hannah@coffee-and-dreams.uk>
2025-06-10 18:33:59 +00:00
Oneric
954f6f7de0 Allow fine-grained announce visibilities
Mastodon supports all standard visibilities except direct here,
but until now we artificially limited this to either fully public
or followers-only.
2025-06-09 20:58:42 +02:00
Oneric
f452430acc Remove non-local recipients from local post addressing
They won’t receive local-only posts anyway
so it makes no sense to list them as recipients.
2025-06-09 20:32:34 +02:00
Oneric
2cdc4acce9 common_api/utils: break up get_to_and_cc
To make it usable in scenarios without a draft.
The next commit adds a user for the new function.

This does technically change behaviour a bit, since
"private" relies to "direct" messages no longer implicitly
address the parent post’s actor, but this seems like a contrived
scenario and was likely never intended to actually occur anyway
as cocorroborated by the absence of tests for it.
2025-06-09 20:31:35 +02:00
Oneric
242e798054 Merge pull request 'Fix Finch timeout config options' (#939) from Oneric/akkoma:finch-timeouts into develop
Reviewed-on: https://akkoma.dev/AkkomaGang/akkoma/pulls/939
2025-06-08 11:11:12 +00:00
Oneric
80a65be38f config: raise Finch pool timeout
A pool timeout shorter than the receive timeout
makes race conditions leading to active connections
being killed more likely and laso just doesn’t make
much sense in general.

See: https://github.com/sneako/finch/pull/292
2025-06-08 12:37:13 +02:00
Oneric
07abbd8de5 config: drop unused duplicate finch pool timeout config
This was added in a924e117fd
with its name mirroring Finch’s own config option, but with
pool_timeout such a setting already existed since
2fe1484ed3 and the new one
was never actually used.
2025-06-08 12:21:33 +02:00
Oneric
3bc583d14f Merge pull request 'Don’t crash due to broken JSON on emoji reload' (#933) from Oneric/akkoma:emoji-reload-no-crash into develop
Reviewed-on: https://akkoma.dev/AkkomaGang/akkoma/pulls/933
2025-06-07 20:01:48 +00:00
Oneric
401883b230 Merge pull request 'provide full replies collection in ActivityPub objects' (#904) from Oneric/akkoma:ap_replies_collection into develop
Reviewed-on: https://akkoma.dev/AkkomaGang/akkoma/pulls/904
2025-06-07 19:33:40 +00:00
Oneric
489b17a60d test/emoji: clean up after failure
Since the cleanup is only registered after everything succeeded,
no cleanup was done on failure permanently breaking the test
2025-06-07 21:22:37 +02:00
Oneric
98762992fb emoji: avoid crashing whole server on broken JSON
It may still crash due to a race condition between checking for file
existence and opening/streaming, but File.stream! has no safe version
we can use to avoid this completely.
Just not deleting such files during a reload is easy enough.
2025-06-07 21:22:37 +02:00
Oneric
b41a13df56 federation/out: add full replies collection to objects
Until now only a limited number of self-replies were inlined as an
anonymous, unordered ActivityPub collection. Notably the advertised
replies might be private posts.

However, providing all (non-private) replies allows for better thread
consistency across instances if the remote server cooperates.
The collection existing as a stndalone object has two advantages
for this. For one, if it was still anonymous, _all_ replies would need
to be inlined, which might be too bloated in pathological cases.
Secondly, it allows remote servers to update the thread by traversing
the reply collection independent of the original post. (If the remote
part knows about chronological ordering, it can in theory
even efficiently resume from where it previously stopped)
2025-06-07 21:02:31 +02:00
Oneric
c55654876f federation/in: handle ordered 'replies' collections
An OrderedCollection uses orderedItems instead of the items key.
So far inlined orderedCollections thus failed to be processed.
Ordered replies collections are used e.g. by IceShrimp.NET and Sharkey,
while Mastodon uses a partially inlined plain Collection.
2025-06-07 21:02:31 +02:00
Oneric
77749bf107 pagination: prevent path params from turning query params
Not all endpoints use OpenAPISpex’ string-to-atom mapping
and they’ll end up with path params being promoted to
query params in pagination next/prev links.
Fix this by never including path params in the first place
2025-06-07 21:02:31 +02:00
Oneric
d1cd3de282 ap/views: factor out common keyset collection bits
The enxt commit will add another user of the new function.
2025-06-07 21:02:31 +02:00
Oneric
80e8c29e64 cosmetic/web/ap/view: split out collection helper functions
And use more precise name.
2025-06-07 21:02:31 +02:00
Oneric
138b1c5165 pagination: fix custom id_type when used with string keys
Ecto.cast is will convert valid string keys to atoms, but can
only deal with inputs which use either string keys everywhere
or atom keys everywhere.
Since :id_type is used before the case it must be an atom,
thus it was impossible to use it with string paramteres before.
2025-06-07 21:02:31 +02:00
Oneric
1ae7d03f5d pagination: support ascending/chronolgical ordering
Up until now queries were always forced into descending ID order
(reverse chronological order with our ID schemes).
Now it’s possible to request the reverse by passing `oder_asc: true`.
2025-06-07 21:02:31 +02:00
Oneric
5c79ff8503 queries: drop unused query function 2025-06-07 21:02:31 +02:00
Oneric
89678f92d9 ap/collection: don't advertise next page if no more entries
And off-by-one error caused the last page
to always advertise one more page eventhough
the absence of further items is known
2025-06-07 21:02:31 +02:00
Oneric
5987dd43d4 Merge pull request 'Rework HTTPSignatures and fix bridgy interop' (#874) from Oneric/akkoma:httpsig_rework into develop
Reviewed-on: https://akkoma.dev/AkkomaGang/akkoma/pulls/874
2025-06-07 19:00:01 +00:00
Oneric
61918d39d2 changelog: summarise user-visible parts of http signature rework 2025-06-07 20:27:58 +02:00
Oneric
09d8de648f mix: point to AkkomaGang repo
The patches are now merged
2025-06-07 20:27:58 +02:00
Oneric
511e49ffa3 Lower load messages for individual emoji packs to debug
The initial info message listing all found packs ought to be sufficient
and with many packs installed thiscan create multiple pages of log
messages on each emoji reload or server start.
Any errors or non-indexed packs are still logged to higher levels.
2025-06-07 20:27:58 +02:00
Oneric
2b885288fa ap: don't require explicit addressing of personal-inbox owners
This requirement was originally added together with splicing the
inbox owner into the non b* addressing fields to make bcc transports
work in https://git.pleroma.social/pleroma/pleroma/-/merge_requests/390.
Later on this was relaxed to always allow deliveries devoid of any
addressing at all in f6cb963df2
and always allow deliveries from actors the owner is following in
750b369d04 to fix interop issues with
Mastodon and Honk respectively.

The justification for both the filtering and splicing comes from
one sentence in AP spec’s inbox section:
> In general, the owner of an inbox is likely
> to be able to access all of their inbox contents.

While this may provide plausible justification for splicing the owner
into cc, it is less clear how this requires or justifies the set of
filtering rules employed here.
Surveying a few other implementations no similar
filtering or splicing appears to be employed.

Furthermore, spec-compliant servers will strip bto/bcc _before_
delivery to remote servers, meaning any compliant bcc transport
out there will NOT contain any explicit addressing of the inbox owner.
Thus the addressing requirement directly opposes
the goal of the original patch.

Currently the requirement for the owner to be addressed once again
is causing interop issues. It turns out to be the root cause of
a long-standing (2+ years) bug preventing meaningful federation.
Bridgy sends e.g. Follow activities and Accepts for Follows directly
to the affected user’s personal inbox while solely addressing
the public scope in the to field. Notably follow relations never
getting established prevented the "accept if followed" allow rule
to ever come into effect.

To make matters worse non-addressed messages simply lead to a
vague "internal server error" response being sent back
which likely slowed down locating the issue.
Furthermore additional issues wrt to signatures cropped up after
the 500-response issues wa first reported, but they seem to have
already been fixed in the meantime, possibly with the signature
handling overhaul in Akkoma.

Given it repeatedly caused issues, does not appear to align with common
practice in the wider fedi ecosystem and apparently contradicts its
original intention, simply remove the requirement.

This is confirmed to fix bridgy interop.

The addressing splicing actually should also add the inbox owner to bto
or bcc instead of cc, but for now this is not changed and in practice
bto/bcc delivery appears to be basically unused anyway.
2025-06-07 20:27:58 +02:00
Oneric
dbce9675e8 Distinguish targeted ratelimit and general unavailability in logs 2025-06-07 20:27:58 +02:00
Oneric
d7bb6551b1 http_signatures: ensure mandatory headers are set
Most headers are automatically checked by the library after this
upgrade. But since digest is only required for requests with a body
and body processing is handled outside the lib atm, we need to
explicity pass the presence or absence along or not get feedback
about creating broken signatures.

This makes bugs in our signatures more apparent
allowing faster discovery and fixing
2025-06-07 20:27:58 +02:00
Oneric
9e52496a20 http_signatures: only compute request-target aliases when needed
Activity db queries are not cached
and most request will not actually need these aliases
2025-06-07 20:27:58 +02:00
Oneric
69a2b4d149 http_signatures: short-circuit gracefully on MRF rejects
And adjust log details
2025-06-07 20:27:58 +02:00
Oneric
5218a7ca2f federation: fake success on Deletes signed with an unknown gone key 2025-06-07 20:27:58 +02:00
Oneric
6e7dee552a federation: let http_signatures library handle request aliases
This avoids spurious key refetches on each failing alias
2025-06-07 20:27:58 +02:00
Oneric
8dad70e8e7 instances: drop has_request_signatures
This property was introduced as a way to gauge whether and
how much enabling authfetch might break passive federation in
https://akkoma.dev/AkkomaGang/akkoma/pulls/312.

However, with the db field defaulting to false, there’s no distinction
between instances without valid signatures and those which just never
attempted to fetch anything from the local instance.
Furthermore, this was never exposed anywhere and required manually
checking the database or cachex state via a remote shell.

Given the above it appears this doesn't actually
provide anything useful, thus drop it.
2025-06-07 20:27:58 +02:00
Oneric
f2ca71f1ad Adapt to new http_signature API 2025-06-07 20:27:58 +02:00
Oneric
fefc884f22 Drop EnsureUserPublicKey plug
It is not needed since fetch_public_keys will already
initiate remote lookup if necessary
2025-06-07 20:27:58 +02:00
Oneric
d53a779c35 Merge pull request 'NormalizeMarkup scrub contentMap' (#932) from Oneric/akkoma:normalize-markup_contentmap into develop
Reviewed-on: https://akkoma.dev/AkkomaGang/akkoma/pulls/932
2025-06-07 18:27:26 +00:00