From mboxrd@z Thu Jan 1 00:00:00 1970 Received: from mail-pf1-x44a.google.com (mail-pf1-x44a.google.com [IPv6:2607:f8b0:4864:20::44a]) by mail.toke.dk (Postfix) with ESMTPS id 7F5A39B2EA0 for ; Fri, 4 Nov 2022 04:25:49 +0100 (CET) Authentication-Results: mail.toke.dk; dkim=pass (2048-bit key; unprotected) header.d=google.com header.i=@google.com header.a=rsa-sha256 header.s=20210112 header.b=oWWEdrBP Received: by mail-pf1-x44a.google.com with SMTP id u3-20020a056a00124300b0056d4ab0c7cbso1691824pfi.7 for ; Thu, 03 Nov 2022 20:25:49 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=20210112; h=cc:to:from:subject:message-id:references:mime-version:in-reply-to :date:from:to:cc:subject:date:message-id:reply-to; bh=/SmIMRH/Y1RlEhU9+aJo9WoglDO0d23IHXz7QGzof4o=; b=oWWEdrBP33iyR3WA+I17MgMvutvdn0/iPBiKK5q4K2Nn/nGxsVPsVPqBrDkhuaaRoW jY0FQELOqD57dEzlX9cYJVgmnRIVQyOHtuAycjNg7z7CyoklXyevcbR7k19/BKhXngFQ L8NHGO2QupByMiipH5OPRb4z8O4umyWVzVXluLWDtkbyF4nZSxO3wJp0DMNI8Kfc57X7 CMH2JRHRzbcj7GzUFxm933O2lA/dPXZw3S+P84EWgB6EegNk4kXb/Mojjny8zbsgJanX mM91Hp2kwt2POgYfk3sKhNu3B4LiQj7Aoe1E67WDul/w0fVoaUATqb6Rz3JQnibSAmRk CFtg== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20210112; h=cc:to:from:subject:message-id:references:mime-version:in-reply-to :date:x-gm-message-state:from:to:cc:subject:date:message-id:reply-to; bh=/SmIMRH/Y1RlEhU9+aJo9WoglDO0d23IHXz7QGzof4o=; b=i9fSBwdo28JQyrwGAlDjDEmfPecv3ShGUivxPwZ5UJ4tGKJqkj2TM0fOW0hIgELzXx YJ6xo7Hg4Uv1Qc6PhEfmVzc/JqAJZzmoMZ1ia4x6WaTWaj3dKFkj0A7UA6QfcxJar3uV 9d4A+pZS7PTsVAC6MMM9nl6Ly9DIP/R2Pg5I3xAz5Mw+MmuBi5YEAPn8BW2jd5Qa+R55 MarOXhp/W99T1XRmb5DTC25KOe6MN3IWfhVGueF5z5JvvqQivVkxwMJ3+VrCbM4sbbe3 9CKoXuPcnzaiC/IjHM1nLTUilSLVWfGoABKXJS57+I2k40JkZXCorcQL57hXP5tDrrUL Tg7w== X-Gm-Message-State: ACrzQf0AW/PSJHSPTOOq513g6LftlKeDqgExuAuEZP8PawH8tk5r30/a OUIsZKYgZWb5FlLL8pyQJTMByww= X-Google-Smtp-Source: AMsMyM4A5OOe8cbq45cAGa7FY0nMjH57LZI4EDuiMJTcLrV982AD3efX0qrTVub/Ev8SMCSkXhjlsvA= X-Received: from sdf.c.googlers.com ([fda3:e722:ac3:cc00:7f:e700:c0a8:5935]) (user=sdf job=sendgmr) by 2002:aa7:8e81:0:b0:56b:9ce2:891f with SMTP id a1-20020aa78e81000000b0056b9ce2891fmr247317pfr.43.1667532347949; Thu, 03 Nov 2022 20:25:47 -0700 (PDT) Date: Thu, 3 Nov 2022 20:25:26 -0700 In-Reply-To: <20221104032532.1615099-1-sdf@google.com> Mime-Version: 1.0 References: <20221104032532.1615099-1-sdf@google.com> X-Mailer: git-send-email 2.38.1.431.g37b22c650d-goog Message-ID: <20221104032532.1615099-9-sdf@google.com> From: Stanislav Fomichev To: bpf@vger.kernel.org Content-Type: text/plain; charset="UTF-8" Message-ID-Hash: QRMTONZITKS3SKXZHFL6JP7GU27PFAFN X-Message-ID-Hash: QRMTONZITKS3SKXZHFL6JP7GU27PFAFN X-MailFrom: 3O4ZkYwMKCfoufhiqqing.eqozfr-jkpvuzfr-rtqlgev.pgv@flex--sdf.bounces.google.com X-Mailman-Rule-Misses: dmarc-mitigation; no-senders; approved; emergency; loop; banned-address; member-moderation; nonmember-moderation; administrivia; implicit-dest; max-recipients; max-size; news-moderation; no-subject; digests; suspicious-header CC: ast@kernel.org, daniel@iogearbox.net, andrii@kernel.org, martin.lau@linux.dev, song@kernel.org, yhs@fb.com, john.fastabend@gmail.com, kpsingh@kernel.org, sdf@google.com, haoluo@google.com, jolsa@kernel.org, David Ahern , Jakub Kicinski , Willem de Bruijn , Jesper Dangaard Brouer , Anatoly Burakov , Alexander Lobakin , Magnus Karlsson , Maryam Tahhan , xdp-hints@xdp-project.net, netdev@vger.kernel.org X-Mailman-Version: 3.3.5 Precedence: list Subject: [xdp-hints] [RFC bpf-next v2 08/14] bpf: Helper to simplify calling kernel routines from unrolled kfuncs List-Id: XDP hardware hints design discussion Archived-At: List-Archive: List-Help: List-Owner: List-Post: List-Subscribe: List-Unsubscribe: When we need to call the kernel function from the unrolled kfunc, we have to take extra care: r6-r9 belong to the callee, not us, so we can't use these registers to stash our r1. We use the same trick we use elsewhere: ask the user to provide extra on-stack storage. Also, note, the program being called has to receive and return the context. Cc: John Fastabend Cc: David Ahern Cc: Martin KaFai Lau Cc: Jakub Kicinski Cc: Willem de Bruijn Cc: Jesper Dangaard Brouer Cc: Anatoly Burakov Cc: Alexander Lobakin Cc: Magnus Karlsson Cc: Maryam Tahhan Cc: xdp-hints@xdp-project.net Cc: netdev@vger.kernel.org Signed-off-by: Stanislav Fomichev --- include/net/xdp.h | 4 ++++ net/core/xdp.c | 24 +++++++++++++++++++++++- 2 files changed, 27 insertions(+), 1 deletion(-) diff --git a/include/net/xdp.h b/include/net/xdp.h index 8c97c6996172..09c05d1da69c 100644 --- a/include/net/xdp.h +++ b/include/net/xdp.h @@ -440,10 +440,14 @@ static inline u32 xdp_metadata_kfunc_id(int id) return xdp_metadata_kfunc_ids.pairs[id].id; } void xdp_metadata_export_to_skb(const struct bpf_prog *prog, struct bpf_patch *patch); +void xdp_kfunc_call_preserving_r1(struct bpf_patch *patch, size_t r0_offset, + void *kfunc); #else #define xdp_metadata_magic 0 static inline u32 xdp_metadata_kfunc_id(int id) { return 0; } static void xdp_metadata_export_to_skb(const struct bpf_prog *prog, struct bpf_patch *patch) { return 0; } +static void xdp_kfunc_call_preserving_r1(struct bpf_patch *patch, size_t r0_offset, + void *kfunc) {} #endif #endif /* __LINUX_NET_XDP_H__ */ diff --git a/net/core/xdp.c b/net/core/xdp.c index 8204fa05c5e9..16dd7850b9b0 100644 --- a/net/core/xdp.c +++ b/net/core/xdp.c @@ -737,6 +737,7 @@ BTF_SET8_START_GLOBAL(xdp_metadata_kfunc_ids) XDP_METADATA_KFUNC_xxx #undef XDP_METADATA_KFUNC BTF_SET8_END(xdp_metadata_kfunc_ids) +EXPORT_SYMBOL(xdp_metadata_kfunc_ids); /* Make sure userspace doesn't depend on our layout by using * different pseudo-generated magic value. @@ -756,7 +757,8 @@ static const struct btf_kfunc_id_set xdp_metadata_kfunc_set = { * * The above also means we _cannot_ easily call any other helper/kfunc * because there is no place for us to preserve our R1 argument; - * existing R6-R9 belong to the callee. + * existing R6-R9 belong to the callee. For the cases where calling into + * the kernel is the only option, see xdp_kfunc_call_preserving_r1. */ void xdp_metadata_export_to_skb(const struct bpf_prog *prog, struct bpf_patch *patch) { @@ -832,6 +834,26 @@ void xdp_metadata_export_to_skb(const struct bpf_prog *prog, struct bpf_patch *p bpf_patch_resolve_jmp(patch); } +EXPORT_SYMBOL(xdp_metadata_export_to_skb); + +/* Helper to generate the bytecode that calls the supplied kfunc. + * The kfunc has to accept a pointer to the context and return the + * same pointer back. The user also has to supply an offset within + * the context to store r0. + */ +void xdp_kfunc_call_preserving_r1(struct bpf_patch *patch, size_t r0_offset, + void *kfunc) +{ + bpf_patch_append(patch, + /* r0 = kfunc(r1); */ + BPF_EMIT_CALL(kfunc), + /* r1 = r0; */ + BPF_MOV64_REG(BPF_REG_1, BPF_REG_0), + /* r0 = *(r1 + r0_offset); */ + BPF_LDX_MEM(BPF_DW, BPF_REG_0, BPF_REG_1, r0_offset), + ); +} +EXPORT_SYMBOL(xdp_kfunc_call_preserving_r1); static int __init xdp_metadata_init(void) { -- 2.38.1.431.g37b22c650d-goog