This allows to retain posts and boosts of remote actors with local
follows regardless of age.
With the "full" setting this can be taken further treating such
followed actors just like local users even keeping all posts they
liked or reacated to.
Pinned objects and their threads will be refetched
on user refresh which by default happens after a day
once a user is encountered again in any form including a mention.
We observed pruning pinned objects usually results in heavy load for
hours after a database prune due to a clogged up remote fetch queue as
pinned posts and their threads of many (most?) users get refetched.
Thus do not prune pinned posts by default.
Keeping closer to earlier behaviour this will still prune threads of
pinned posts regardless of --keep-threads if nothing else prevenets it.
In theory a pedantic reading of the spec indeed suggests
DMs must only be delivered to personal inboxes. However,
in practice the normative force of real-world implementations
disagrees. Mastodon, Iceshrimp.NET and GtS (the latter notably has a
config option to never use sharedInboxes) all unconditionally prefer
sharedInbox for everything without ill effect. This saves on duplicate
deliveries on the sending and processing on the receiving end.
(Typically the receiving side ends up rejecting
all but the first copy as duplicates)
Furthermore current determine_inbox logic also actually needs up
forcing personal inboxes for follower-only posts, unless they
additionally explicitly address at least one specific actor.
This is even much wasteful and directly contradicts
the explicit intent of the spec.
There’s one part where the use of sharedInbox falls apart,
namely spec-compliant bcc and bto addressing. AP spec requires
bcc/bto fields to be stripped before delivery and then implicitly
reconstructed by the receiver based on the addressed personal inbox.
In practice however, this addressing mode is almost unused. Neither of
the three implementations brought up above supports it and while *oma
does use bcc for list addressing, it does not use it in a spec-compliant
way and even copies same-host recipients into cc before delivery.
Messages with bcc addressing are handled in another function clause,
always force personal inboxes for every recipient and not affected by
this commit.
In theory it would be beneficial to use sharedInbox there too for all
but bcc recipients. But in practice list addressing has been broken for
quite some time already and is not actually exposed in any frontend,
as discussed in https://akkoma.dev/AkkomaGang/akkoma/issues/812.
Therefore any changes here have virtually no effect anyway
and all code concerning it may just be outright removed.
This allows discovering a page represents an ActivityPub object
and also where to find the underlying representation.
Other servers already implement this and some tools
came to rely or profit from it.
The alternate link is provided both with the "application/activity+json"
format as used by Mastodon and the standard-compliant media type.
Just like the feed provider, ActivityPub links are always enabled
unless access to local posts is restricted and not configurable.
The commit is based on earlier work by Charlotte 🦝 Deleńkec
but with fixes and some tweaks.
Co-authored-by: Charlotte 🦝 Deleńkec <lotte@chir.rs>
:discard marks jobs as "discarded", i.e. jobs which permanently failed
due to e.g. exhausting all retries or explicitly being discared due to a
fatal error.
:cancel marks jobs as "cancelled" which does not imply failure.
While neither method counts as a job "exception" in the set of
telemetries we currently export via Prometheus, the different state
is visible in the (not-exported) metadata of oban job telemetry.
We can use handlers of those events to build bespoke statistics.
Ideally we'd like to distinguish in the receiver worker between
"invalid" and "already present or delete of unknown" documents,
but this is cumbersome to get get right with a list of
free-form, human-readable descriptions oof the violated constraints.
For now, just count both as an fatal error.
# but that is cumbersome to get right with a list of string error descriptions
CUrrently internal actors are supposed to be identified in the database
by either a NULL nickname or a nickname prefixed by "internal.". For old
installations this is true, but only if they were created over five
years ago before 70410dfafd.
Newer installations will use "relay" as the nickname of the realy actor
causing ii to be treated as a regular user.
In particular this means all installations in the last five years never
made use of the reduced endpoint case, thus it is dropped.
Simplify this distinction by properly marking internal actors asa an
Application type in the database. This was already implemented before by
ilja in https://akkoma.dev/AkkomaGang/akkoma/pulls/457 but accidentally
reverted during a translation update in
eba3cce77b. This commit effectively
restores this patch together with further changes.
Also service actors unconditionally expose follow* collections atm,
eventhough the internal fetch actor doesn't actually implement them.
Since they are optional per spec and with Mastodon omitting them too
for its instance actor proving the practical viability, we should just
omit them. The relay actor however should continue to expose such
collections and they are properly implemented here.
Here too we now just use the values or their absence in the database.
We do not have any other internal.* actors besides fetch atm.
Fixes: https://akkoma.dev/AkkomaGang/akkoma/issues/855
Co-authored-by: ilja space <git@ilja.space>
While the object itself has the expected adressing for an
"unlisted" post, we always use the Create activity’s
adressing fields for permission checks.
To avoid unintended effects on legacy objects
we will continue to use the activity for access perm checks,
but fix its addressing fields based on its object data.
Ref: https://git.pleroma.social/pleroma/pleroma/-/issues/3323
This was accidentally broken in c8e0f7848b
due to a one-letter mistake in the plug option name and an absence of
tests. Therefore it was once again possible to serve e.g. Javascript or
CSS payloads via uploads and emoji.
However due to other protections it was still NOT possible for anyone to
serve any payload with an ActivityPub Content-Type. With the CSP policy
hardening from previous JS payload exloits predating the Content-Type
sanitisation, there is currently no known way of abusing this weakened
Content-Type sanitisation, but should be fixed regardless.
This commit fixes the option name and adds tests to ensure
such a regression doesn't occur again in the future.
Reported-by: Lain Soykaf <lain@lain.com>
When note editing support was added, it was omitted to strip internal
fields from edited notes and their history.
This was uncovered due to Mastodon inlining the like count as a "likes"
collection conflicting with our internal "likes" list causing validation
failures. In a spot check with likes/like_count it was not possible to
inject those internal fields into the local db via Update, but this
was not extensively tested for all fields and avenues.
Similarly address normalisation did not normalise addressing in the
object history, although this was never at risk of being exploitable.
The revision history of the Pleroma MR adding edit support reveals
recusrive stripping was intentionally avoided, since it will end up
removing e.g. emoji from outgoing activities. This appears to still
be true. However, all current internal fields ("pleroma_interal"
appears to be unused) contain data already publicised otherwise anyway.
In the interest of fixing a federation bug (and at worst potential data
injection) quickly outgoing stripping is left non-recursive for now.
Of course the ultimate fix here is to not mix remote and internal data
into the same map in the first place, but unfortunately having a single
map of all truth is a core assumption of *oma's AP doc processing.
Changing this is a masive undertaking and not suitable for providing
a short-term fix.
This matches behaviour prioir to the SigningKey migration
and the expected semantics of the http_signatures lib.
Additionally add a min interval paramter, to avoid
refetch floods on bugs causing incompatible signatures
(like e.g. currently with Bridgy)
User updates broke with the migration to separate signing keys
since user data carries signing keys but we didn't allow the
association data to be updated.
Previously there were mainly two attack vectors:
- for raw keys the owner <-> key mapping wasn't verified at all
- keys were retrieved with refetching allowed
and only the top-level ID was sanitised while
usually keys are but a subobject
This reintroduces public key checks in the user actor,
previously removed in 9728e2f8f7
but now adapted to account for the new mapping mechanism.
User.get_or_fetch_by_(apid|nickname) are the only external users of fetch_and_prepare_user_from_ap_id,
thus there’s no point in duplicating logging, expecially not at error level.
Currently (duplicated) _not_found errors for users make up the bulk of my logs
and are created almost every second. Deleted users are a common occurence and not
worth logging outside of debug
This query is one of the top cost offenders during an instances
lifetime. For small instances it was shown to take up 30-50% percent of
the total database query time, while for bigger isntaces it still held
a spot in the top 3 — alost as or even more expensive overall than
timeline queries!
The good news is, there’s a cheaper way using the instance table:
no need to process each entry, no need to filter NULLs
and no need to dedupe. EXPLAIN estimates the cost of the
old query as 13272.39 and the cost of the new query as 395.74
for me; i.e. a 33-fold reduction.
Results can slightly differ. E.g. we might have an old user
predating the instance tables existence and no interaction with since
or no instance table entry due to failure to query nodeinfo.
Conversely, we might have an instance entry but all known users got
deleted since.
However, this seems unproblematic in practice
and well worth the perf improvment.
Given the previous query didn’t exclude unreachable instances
neither does the new query.
There were two issues leading to needles effort:
Most importnatly, the use of AP IDs as "source_url" meant multiple
simultaneous jobs got scheduled for the same instance even with the
default unique settings.
Also jobs were scheduled uncontionally for each processed AP object
meaning we incured oberhead from managing Oban jobs even if we knew it
wasn't necessary. By comparison the single query to check if an update
is needed should be cheaper overall.
It was used to migrate OStatus connections to ActivityPub if possible,
but support for OStatus was long since dropped, all new actors always AP
and if anything wasn't migrated before, their instance is already marked
as unreachable anyway.
The associated logic was also buggy in several ways and deleted users
got set to ap_enabled=false also causing some issues.
This patch is a pretty direct port of the original Pleroma MR;
follow-up commits will further fix and clean up remaining issues.
Changes made (other than trivial merge conflict resolutions):
- converted CHANGELOG format
- adapted migration id for Akkoma’s timeline
- removed ap_enabled from additional tests
Ported-from: https://git.pleroma.social/pleroma/pleroma/-/merge_requests/3880
Otherwise attachments have a high chance to disappear with akkoma-fe’s
“delete & redraft” feature when cleanup is enabled in the backend. Since
we don't know whether a deletion was intended to be part of a redraft
process or even if whether the redraft was abandoned we still have to
delete attachments eventually.
A thirty minute delay should provide sufficient time for redrafting.
Fixes: https://akkoma.dev/AkkomaGang/akkoma/issues/775
