Merge remote-tracking branch 'remotes/origin/develop' into 1505-threads-federation

# Conflicts:
#	CHANGELOG.md
#	config/config.exs
This commit is contained in:
Ivan Tashkinov 2020-02-22 09:31:43 +03:00
commit 8f0ca19b9c
17 changed files with 209 additions and 58 deletions

View File

@ -74,6 +74,7 @@ The format is based on [Keep a Changelog](https://keepachangelog.com/en/1.0.0/).
- User settings: Add _This account is a_ option. - User settings: Add _This account is a_ option.
- A new users admin digest email - A new users admin digest email
- OAuth: admin scopes support (relevant setting: `[:auth, :enforce_oauth_admin_scope_usage]`). - OAuth: admin scopes support (relevant setting: `[:auth, :enforce_oauth_admin_scope_usage]`).
- Add an option `authorized_fetch_mode` to require HTTP signatures for AP fetches.
- ActivityPub: support for `replies` collection (output for outgoing federation & fetching on incoming federation). - ActivityPub: support for `replies` collection (output for outgoing federation & fetching on incoming federation).
<details> <details>
<summary>API Changes</summary> <summary>API Changes</summary>
@ -116,6 +117,7 @@ The format is based on [Keep a Changelog](https://keepachangelog.com/en/1.0.0/).
- Configuration: `feed.logo` option for tag feed. - Configuration: `feed.logo` option for tag feed.
- Tag feed: `/tags/:tag.rss` - list public statuses by hashtag. - Tag feed: `/tags/:tag.rss` - list public statuses by hashtag.
- Mastodon API: Add `reacted` property to `emoji_reactions` - Mastodon API: Add `reacted` property to `emoji_reactions`
- Pleroma API: Add reactions for a single emoji.
- ActivityPub: `[:activitypub, :note_replies_output_limit]` setting sets the number of note self-replies to output on outgoing federation. - ActivityPub: `[:activitypub, :note_replies_output_limit]` setting sets the number of note self-replies to output on outgoing federation.
</details> </details>

View File

@ -327,7 +327,8 @@
outgoing_blocks: true, outgoing_blocks: true,
follow_handshake_timeout: 500, follow_handshake_timeout: 500,
note_replies_output_limit: 5, note_replies_output_limit: 5,
sign_object_fetches: true sign_object_fetches: true,
authorized_fetch_mode: false
config :pleroma, :streamer, config :pleroma, :streamer,
workers: 3, workers: 3,
@ -618,6 +619,8 @@
config :pleroma, configurable_from_database: false config :pleroma, configurable_from_database: false
config :pleroma, Pleroma.Repo, parameters: [gin_fuzzy_search_limit: "500"]
# Import environment specific config. This must remain at the bottom # Import environment specific config. This must remain at the bottom
# of this file so it overrides the configuration defined above. # of this file so it overrides the configuration defined above.
import_config "#{Mix.env()}.exs" import_config "#{Mix.env()}.exs"

View File

@ -459,3 +459,16 @@ Emoji reactions work a lot like favourites do. They make it possible to react to
{"name": "☕", "count": 1, "me": false, "accounts": [{"id" => "abc..."}]} {"name": "☕", "count": 1, "me": false, "accounts": [{"id" => "abc..."}]}
] ]
``` ```
## `GET /api/v1/pleroma/statuses/:id/reactions/:emoji`
### Get an object of emoji to account mappings with accounts that reacted to the post for a specific emoji`
* Method: `GET`
* Authentication: optional
* Params: None
* Response: JSON, a list of emoji/account list tuples
* Example Response:
```json
[
{"name": "😀", "count": 2, "me": true, "accounts": [{"id" => "xyz.."...}, {"id" => "zyx..."}]}
]
```

View File

@ -18,7 +18,9 @@
6. Run `sudo -Hu postgres pg_restore -d <pleroma_db> -v -1 </path/to/backup_location/pleroma.pgdump>` 6. Run `sudo -Hu postgres pg_restore -d <pleroma_db> -v -1 </path/to/backup_location/pleroma.pgdump>`
7. If you installed a newer Pleroma version, you should run `mix ecto.migrate`[^1]. This task performs database migrations, if there were any. 7. If you installed a newer Pleroma version, you should run `mix ecto.migrate`[^1]. This task performs database migrations, if there were any.
8. Restart the Pleroma service. 8. Restart the Pleroma service.
9. After you've restarted Pleroma, you will notice that postgres will take up more cpu resources than usual. A lot in fact. To fix this you must do a VACUUM ANLAYZE. This can also be done while the instance is still running like so:
$ sudo -u postgres psql pleroma_database_name
pleroma=# VACUUM ANALYZE;
[^1]: Prefix with `MIX_ENV=prod` to run it using the production config file. [^1]: Prefix with `MIX_ENV=prod` to run it using the production config file.
## Remove ## Remove

View File

@ -1,4 +1,21 @@
# Updating your instance # Updating your instance
You should **always check the release notes/changelog** in case there are config deprecations, special update special update steps, etc.
Besides that, doing the following is generally enough:
## For OTP installations
```sh
# Download the new release
su pleroma -s $SHELL -lc "./bin/pleroma_ctl update"
# Migrate the database, you are advised to stop the instance before doing that
su pleroma -s $SHELL -lc "./bin/pleroma_ctl migrate"
```
## For from source installations (using git)
1. Go to the working directory of Pleroma (default is `/opt/pleroma`) 1. Go to the working directory of Pleroma (default is `/opt/pleroma`)
2. Run `git pull`. This pulls the latest changes from upstream. 2. Run `git pull`. This pulls the latest changes from upstream.
3. Run `mix deps.get`. This pulls in any new dependencies. 3. Run `mix deps.get`. This pulls in any new dependencies.

View File

@ -143,10 +143,11 @@ config :pleroma, :mrf_user_allowlist,
* `:reject` rejects the message entirely * `:reject` rejects the message entirely
### :activitypub ### :activitypub
* ``unfollow_blocked``: Whether blocks result in people getting unfollowed * `unfollow_blocked`: Whether blocks result in people getting unfollowed
* ``outgoing_blocks``: Whether to federate blocks to other instances * `outgoing_blocks`: Whether to federate blocks to other instances
* ``deny_follow_blocked``: Whether to disallow following an account that has blocked the user in question * `deny_follow_blocked`: Whether to disallow following an account that has blocked the user in question
* ``sign_object_fetches``: Sign object fetches with HTTP signatures * `sign_object_fetches`: Sign object fetches with HTTP signatures
* `authorized_fetch_mode`: Require HTTP signatures for AP fetches
### :fetch_initial_posts ### :fetch_initial_posts
* `enabled`: if enabled, when a new user is federated with, fetch some of their latest posts * `enabled`: if enabled, when a new user is federated with, fetch some of their latest posts

View File

@ -259,19 +259,14 @@ su pleroma -s $SHELL -lc "./bin/pleroma_ctl user new joeuser joeuser@sld.tld --a
``` ```
This will create an account withe the username of 'joeuser' with the email address of joeuser@sld.tld, and set that user's account as an admin. This will result in a link that you can paste into the browser, which logs you in and enables you to set the password. This will create an account withe the username of 'joeuser' with the email address of joeuser@sld.tld, and set that user's account as an admin. This will result in a link that you can paste into the browser, which logs you in and enables you to set the password.
### Updating
Generally, doing the following is enough:
```sh
# Download the new release
su pleroma -s $SHELL -lc "./bin/pleroma_ctl update"
# Migrate the database, you are advised to stop the instance before doing that
su pleroma -s $SHELL -lc "./bin/pleroma_ctl migrate"
```
But you should **always check the release notes/changelog** in case there are config deprecations, special update steps, etc.
## Further reading ## Further reading
* [Backup your instance](../administration/backup.md) * [Backup your instance](../administration/backup.md)
* [Hardening your instance](../configuration/hardening.md) * [Hardening your instance](../configuration/hardening.md)
* [How to activate mediaproxy](../configuration/howto_mediaproxy.md) * [How to activate mediaproxy](../configuration/howto_mediaproxy.md)
* [Updating your instance](../administration/updating.md)
## Questions
Questions about the installation or didnt it work as it should be, ask in [#pleroma:matrix.org](https://matrix.heldscal.la/#/room/#freenode_#pleroma:matrix.org) or IRC Channel **#pleroma** on **Freenode**.

View File

@ -4,6 +4,7 @@
defmodule Pleroma.Web.Plugs.HTTPSignaturePlug do defmodule Pleroma.Web.Plugs.HTTPSignaturePlug do
import Plug.Conn import Plug.Conn
import Phoenix.Controller, only: [get_format: 1, text: 2]
require Logger require Logger
def init(options) do def init(options) do
@ -15,26 +16,28 @@ def call(%{assigns: %{valid_signature: true}} = conn, _opts) do
end end
def call(conn, _opts) do def call(conn, _opts) do
headers = get_req_header(conn, "signature") if get_format(conn) == "activity+json" do
signature = Enum.at(headers, 0)
if signature do
# set (request-target) header to the appropriate value
# we also replace the digest header with the one we computed
conn =
conn conn
|> put_req_header( |> maybe_assign_valid_signature()
"(request-target)", |> maybe_require_signature()
String.downcase("#{conn.method}") <> " #{conn.request_path}"
)
conn =
if conn.assigns[:digest] do
conn
|> put_req_header("digest", conn.assigns[:digest])
else else
conn conn
end end
end
defp maybe_assign_valid_signature(conn) do
if has_signature_header?(conn) do
# set (request-target) header to the appropriate value
# we also replace the digest header with the one we computed
request_target = String.downcase("#{conn.method}") <> " #{conn.request_path}"
conn =
conn
|> put_req_header("(request-target)", request_target)
|> case do
%{assigns: %{digest: digest}} = conn -> put_req_header(conn, "digest", digest)
conn -> conn
end
assign(conn, :valid_signature, HTTPSignatures.validate_conn(conn)) assign(conn, :valid_signature, HTTPSignatures.validate_conn(conn))
else else
@ -42,4 +45,21 @@ def call(conn, _opts) do
conn conn
end end
end end
defp has_signature_header?(conn) do
conn |> get_req_header("signature") |> Enum.at(0, false)
end
defp maybe_require_signature(%{assigns: %{valid_signature: true}} = conn), do: conn
defp maybe_require_signature(conn) do
if Pleroma.Config.get([:activitypub, :authorized_fetch_mode], false) do
conn
|> put_status(:unauthorized)
|> text("Request not signed")
|> halt()
else
conn
end
end
end end

View File

@ -41,13 +41,16 @@ defmodule Pleroma.Web.PleromaAPI.PleromaAPIController do
plug(Pleroma.Plugs.EnsurePublicOrAuthenticatedPlug) plug(Pleroma.Plugs.EnsurePublicOrAuthenticatedPlug)
def emoji_reactions_by(%{assigns: %{user: user}} = conn, %{"id" => activity_id}) do def emoji_reactions_by(%{assigns: %{user: user}} = conn, %{"id" => activity_id} = params) do
with %Activity{} = activity <- Activity.get_by_id_with_object(activity_id), with %Activity{} = activity <- Activity.get_by_id_with_object(activity_id),
%Object{data: %{"reactions" => emoji_reactions}} when is_list(emoji_reactions) <- %Object{data: %{"reactions" => emoji_reactions}} when is_list(emoji_reactions) <-
Object.normalize(activity) do Object.normalize(activity) do
reactions = reactions =
emoji_reactions emoji_reactions
|> Enum.map(fn [emoji, user_ap_ids] -> |> Enum.map(fn [emoji, user_ap_ids] ->
if params["emoji"] && params["emoji"] != emoji do
nil
else
users = users =
Enum.map(user_ap_ids, &User.get_cached_by_ap_id/1) Enum.map(user_ap_ids, &User.get_cached_by_ap_id/1)
|> Enum.filter(& &1) |> Enum.filter(& &1)
@ -58,7 +61,9 @@ def emoji_reactions_by(%{assigns: %{user: user}} = conn, %{"id" => activity_id})
accounts: AccountView.render("index.json", %{users: users, for: user, as: :user}), accounts: AccountView.render("index.json", %{users: users, for: user, as: :user}),
me: !!(user && user.ap_id in user_ap_ids) me: !!(user && user.ap_id in user_ap_ids)
} }
end
end) end)
|> Enum.filter(& &1)
conn conn
|> json(reactions) |> json(reactions)

View File

@ -271,6 +271,7 @@ defmodule Pleroma.Web.Router do
scope "/api/v1/pleroma", Pleroma.Web.PleromaAPI do scope "/api/v1/pleroma", Pleroma.Web.PleromaAPI do
pipe_through(:api) pipe_through(:api)
get("/statuses/:id/reactions/:emoji", PleromaAPIController, :emoji_reactions_by)
get("/statuses/:id/reactions", PleromaAPIController, :emoji_reactions_by) get("/statuses/:id/reactions", PleromaAPIController, :emoji_reactions_by)
end end

View File

@ -1,5 +1,5 @@
# Pleroma: A lightweight social networking server # Pleroma: A lightweight social networking server
# Copyright © 2017-2019 Pleroma Authors <https://pleroma.social/> # Copyright © 2017-2020 Pleroma Authors <https://pleroma.social/>
# SPDX-License-Identifier: AGPL-3.0-only # SPDX-License-Identifier: AGPL-3.0-only
defmodule Pleroma.Web.TwitterAPI.RemoteFollowController do defmodule Pleroma.Web.TwitterAPI.RemoteFollowController do
@ -69,7 +69,7 @@ defp is_status?(acct) do
def do_follow(%{assigns: %{user: %User{} = user}} = conn, %{"user" => %{"id" => id}}) do def do_follow(%{assigns: %{user: %User{} = user}} = conn, %{"user" => %{"id" => id}}) do
with {:fetch_user, %User{} = followee} <- {:fetch_user, User.get_cached_by_id(id)}, with {:fetch_user, %User{} = followee} <- {:fetch_user, User.get_cached_by_id(id)},
{:ok, _, _, _} <- CommonAPI.follow(user, followee) do {:ok, _, _, _} <- CommonAPI.follow(user, followee) do
render(conn, "followed.html", %{error: false}) redirect(conn, to: "/users/#{followee.id}")
else else
error -> error ->
handle_follow_error(conn, error) handle_follow_error(conn, error)
@ -80,7 +80,7 @@ def do_follow(conn, %{"authorization" => %{"name" => _, "password" => _, "id" =>
with {:fetch_user, %User{} = followee} <- {:fetch_user, User.get_cached_by_id(id)}, with {:fetch_user, %User{} = followee} <- {:fetch_user, User.get_cached_by_id(id)},
{_, {:ok, user}, _} <- {:auth, Authenticator.get_user(conn), followee}, {_, {:ok, user}, _} <- {:auth, Authenticator.get_user(conn), followee},
{:ok, _, _, _} <- CommonAPI.follow(user, followee) do {:ok, _, _, _} <- CommonAPI.follow(user, followee) do
render(conn, "followed.html", %{error: false}) redirect(conn, to: "/users/#{followee.id}")
else else
error -> error ->
handle_follow_error(conn, error) handle_follow_error(conn, error)

View File

@ -17,7 +17,11 @@ def up do
Repo.stream(query) Repo.stream(query)
|> Enum.each(fn %{id: user_id, bookmarks: bookmarks} -> |> Enum.each(fn %{id: user_id, bookmarks: bookmarks} ->
Enum.each(bookmarks, fn ap_id -> Enum.each(bookmarks, fn ap_id ->
activity = Activity.get_create_by_object_ap_id(ap_id) activity =
ap_id
|> Activity.create_by_object_ap_id()
|> Repo.one()
unless is_nil(activity), do: {:ok, _} = Bookmark.create(user_id, activity.id) unless is_nil(activity), do: {:ok, _} = Bookmark.create(user_id, activity.id)
end) end)
end) end)

View File

@ -1,7 +1,8 @@
defmodule Pleroma.Repo.Migrations.AddFollowingAddressFromSourceData do defmodule Pleroma.Repo.Migrations.AddFollowingAddressFromSourceData do
use Ecto.Migration
import Ecto.Query
alias Pleroma.User alias Pleroma.User
import Ecto.Query
require Logger
use Ecto.Migration
def change do def change do
query = query =
@ -19,6 +20,9 @@ def change do
:following_address :following_address
]) ])
|> Pleroma.Repo.update() |> Pleroma.Repo.update()
user ->
Logger.warn("User #{user.id} / #{user.nickname} does not seem to have source_data")
end) end)
end end
end end

View File

@ -2,6 +2,8 @@ defmodule Pleroma.Repo.Migrations.CopyMutedToMutedNotifications do
use Ecto.Migration use Ecto.Migration
def change do def change do
execute("update users set info = '{}' where info is null")
execute( execute(
"update users set info = safe_jsonb_set(info, '{muted_notifications}', info->'mutes', true) where local = true" "update users set info = safe_jsonb_set(info, '{muted_notifications}', info->'mutes', true) where local = true"
) )

View File

@ -7,6 +7,7 @@ defmodule Pleroma.Web.Plugs.HTTPSignaturePlugTest do
alias Pleroma.Web.Plugs.HTTPSignaturePlug alias Pleroma.Web.Plugs.HTTPSignaturePlug
import Plug.Conn import Plug.Conn
import Phoenix.Controller, only: [put_format: 2]
import Mock import Mock
test "it call HTTPSignatures to check validity if the actor sighed it" do test "it call HTTPSignatures to check validity if the actor sighed it" do
@ -20,10 +21,69 @@ test "it call HTTPSignatures to check validity if the actor sighed it" do
"signature", "signature",
"keyId=\"http://mastodon.example.org/users/admin#main-key" "keyId=\"http://mastodon.example.org/users/admin#main-key"
) )
|> put_format("activity+json")
|> HTTPSignaturePlug.call(%{}) |> HTTPSignaturePlug.call(%{})
assert conn.assigns.valid_signature == true assert conn.assigns.valid_signature == true
assert conn.halted == false
assert called(HTTPSignatures.validate_conn(:_)) assert called(HTTPSignatures.validate_conn(:_))
end end
end end
describe "requires a signature when `authorized_fetch_mode` is enabled" do
setup do
Pleroma.Config.put([:activitypub, :authorized_fetch_mode], true)
on_exit(fn ->
Pleroma.Config.put([:activitypub, :authorized_fetch_mode], false)
end)
params = %{"actor" => "http://mastodon.example.org/users/admin"}
conn = build_conn(:get, "/doesntmattter", params) |> put_format("activity+json")
[conn: conn]
end
test "when signature header is present", %{conn: conn} do
with_mock HTTPSignatures, validate_conn: fn _ -> false end do
conn =
conn
|> put_req_header(
"signature",
"keyId=\"http://mastodon.example.org/users/admin#main-key"
)
|> HTTPSignaturePlug.call(%{})
assert conn.assigns.valid_signature == false
assert conn.halted == true
assert conn.status == 401
assert conn.state == :sent
assert conn.resp_body == "Request not signed"
assert called(HTTPSignatures.validate_conn(:_))
end
with_mock HTTPSignatures, validate_conn: fn _ -> true end do
conn =
conn
|> put_req_header(
"signature",
"keyId=\"http://mastodon.example.org/users/admin#main-key"
)
|> HTTPSignaturePlug.call(%{})
assert conn.assigns.valid_signature == true
assert conn.halted == false
assert called(HTTPSignatures.validate_conn(:_))
end
end
test "halts the connection when `signature` header is not present", %{conn: conn} do
conn = HTTPSignaturePlug.call(conn, %{})
assert conn.assigns[:valid_signature] == nil
assert conn.halted == true
assert conn.status == 401
assert conn.state == :sent
assert conn.resp_body == "Request not signed"
end
end
end end

View File

@ -96,6 +96,32 @@ test "GET /api/v1/pleroma/statuses/:id/reactions", %{conn: conn} do
result result
end end
test "GET /api/v1/pleroma/statuses/:id/reactions/:emoji", %{conn: conn} do
user = insert(:user)
other_user = insert(:user)
{:ok, activity} = CommonAPI.post(user, %{"status" => "#cofe"})
result =
conn
|> get("/api/v1/pleroma/statuses/#{activity.id}/reactions/🎅")
|> json_response(200)
assert result == []
{:ok, _, _} = CommonAPI.react_with_emoji(activity.id, other_user, "🎅")
{:ok, _, _} = CommonAPI.react_with_emoji(activity.id, other_user, "")
result =
conn
|> get("/api/v1/pleroma/statuses/#{activity.id}/reactions/🎅")
|> json_response(200)
[%{"name" => "🎅", "count" => 1, "accounts" => [represented_user], "me" => false}] = result
assert represented_user["id"] == other_user.id
end
test "/api/v1/pleroma/conversations/:id" do test "/api/v1/pleroma/conversations/:id" do
user = insert(:user) user = insert(:user)
%{user: other_user, conn: conn} = oauth_access(["read:statuses"]) %{user: other_user, conn: conn} = oauth_access(["read:statuses"])

View File

@ -1,5 +1,5 @@
# Pleroma: A lightweight social networking server # Pleroma: A lightweight social networking server
# Copyright © 2017-2019 Pleroma Authors <https://pleroma.social/> # Copyright © 2017-2020 Pleroma Authors <https://pleroma.social/>
# SPDX-License-Identifier: AGPL-3.0-only # SPDX-License-Identifier: AGPL-3.0-only
defmodule Pleroma.Web.TwitterAPI.RemoteFollowControllerTest do defmodule Pleroma.Web.TwitterAPI.RemoteFollowControllerTest do
@ -92,15 +92,13 @@ test "follows user", %{conn: conn} do
user = insert(:user) user = insert(:user)
user2 = insert(:user) user2 = insert(:user)
response = conn =
conn conn
|> assign(:user, user) |> assign(:user, user)
|> assign(:token, insert(:oauth_token, user: user, scopes: ["write:follows"])) |> assign(:token, insert(:oauth_token, user: user, scopes: ["write:follows"]))
|> post(remote_follow_path(conn, :do_follow), %{"user" => %{"id" => user2.id}}) |> post(remote_follow_path(conn, :do_follow), %{"user" => %{"id" => user2.id}})
|> response(200)
assert response =~ "Account followed!" assert redirected_to(conn) == "/users/#{user2.id}"
assert user2.follower_address in User.following(user)
end end
test "returns error when user is deactivated", %{conn: conn} do test "returns error when user is deactivated", %{conn: conn} do
@ -149,14 +147,13 @@ test "returns success result when user already in followers", %{conn: conn} do
user2 = insert(:user) user2 = insert(:user)
{:ok, _, _, _} = CommonAPI.follow(user, user2) {:ok, _, _, _} = CommonAPI.follow(user, user2)
response = conn =
conn conn
|> assign(:user, refresh_record(user)) |> assign(:user, refresh_record(user))
|> assign(:token, insert(:oauth_token, user: user, scopes: ["write:follows"])) |> assign(:token, insert(:oauth_token, user: user, scopes: ["write:follows"]))
|> post(remote_follow_path(conn, :do_follow), %{"user" => %{"id" => user2.id}}) |> post(remote_follow_path(conn, :do_follow), %{"user" => %{"id" => user2.id}})
|> response(200)
assert response =~ "Account followed!" assert redirected_to(conn) == "/users/#{user2.id}"
end end
end end
@ -165,14 +162,13 @@ test "follows", %{conn: conn} do
user = insert(:user) user = insert(:user)
user2 = insert(:user) user2 = insert(:user)
response = conn =
conn conn
|> post(remote_follow_path(conn, :do_follow), %{ |> post(remote_follow_path(conn, :do_follow), %{
"authorization" => %{"name" => user.nickname, "password" => "test", "id" => user2.id} "authorization" => %{"name" => user.nickname, "password" => "test", "id" => user2.id}
}) })
|> response(200)
assert response =~ "Account followed!" assert redirected_to(conn) == "/users/#{user2.id}"
assert user2.follower_address in User.following(user) assert user2.follower_address in User.following(user)
end end