Skip to content

Commit

Permalink
Merge branch 'net-smcv2.1-ISM-device-support'
Browse files Browse the repository at this point in the history
Wen Gu says:

====================
net/smc: implement SMCv2.1 virtual ISM device support

The fourth edition of SMCv2 adds the SMC version 2.1 feature updates for
SMC-Dv2 with virtual ISM. Virtual ISM are created and supported mainly by
OS or hypervisor software, comparable to IBM ISM which is based on platform
firmware or hardware.

With the introduction of virtual ISM, SMCv2.1 makes some updates:

- Introduce feature bitmask to indicate supplemental features.
- Reserve a range of CHIDs for virtual ISM.
- Support extended GIDs (128 bits) in CLC handshake.

So this patch set aims to implement these updates in Linux kernel. And it
acts as the first part of SMC-D virtual ISM extension & loopback-ism [1].

[1] https://lore.kernel.org/netdev/1695568613-125057-1-git-send-email-guwen@linux.alibaba.com/

v8->v7:
- Patch #7: v7 mistakenly changed the type of gid_ext in
  smc_clc_msg_accept_confirm to u64 instead of __be64 as previous versions
  when fixing the rebase conflicts. So fix this mistake.

v7->v6:
Link: https://lore.kernel.org/netdev/20231219084536.8158-1-guwen@linux.alibaba.com/
- Collect the Reviewed-by tag in v6;
- Patch #3: redefine the struct smc_clc_msg_accept_confirm;
- Patch #7: Because that the Patch #3 already adds '__packed' to
  smc_clc_msg_accept_confirm, so Patch #7 doesn't need to do the same thing.
  But this is a minor change, so I kept the 'Reviewed-by' tag.

Other changes in previous versions but not yet acked:
- Patch #1: Some minor changes in subject and fix the format issue
  (length exceeds 80 columns) compared to v3.
- Patch #5: removes useless ini->feature_mask assignment in __smc_connect()
  and smc_listen_v2_check() compared to v4.
- Patch #8: new added, compared to v3.

v6->v5:
Link: https://lore.kernel.org/netdev/1702371151-125258-1-git-send-email-guwen@linux.alibaba.com/
- Add 'Reviewed-by' label given in the previous versions:
  * Patch #4, #6, #9, #10 have nothing changed since v3;
- Patch #2:
  * fix the format issue (Alignment should match open parenthesis) compared to v5;
  * remove useless clc->hdr.length assignment in smcr_clc_prep_confirm_accept()
    compared to v5;
- Patch #3: new added compared to v5.
- Patch #7: some minor changes like aclc_v2->aclc or clc_v2->clc compared to v5
  due to the introduction of Patch #3. Since there were no major changes, I kept
  the 'Reviewed-by' label.

Other changes in previous versions but not yet acked:
- Patch #1: Some minor changes in subject and fix the format issue
  (length exceeds 80 columns) compared to v3.
- Patch #5: removes useless ini->feature_mask assignment in __smc_connect()
  and smc_listen_v2_check() compared to v4.
- Patch #8: new added, compared to v3.

v5->v4:
Link: https://lore.kernel.org/netdev/1702021259-41504-1-git-send-email-guwen@linux.alibaba.com/
- Patch #6: improve the comment of SMCD_CLC_MAX_V2_GID_ENTRIES;
- Patch #4: remove useless ini->feature_mask assignment;

v4->v3:
https://lore.kernel.org/netdev/1701920994-73705-1-git-send-email-guwen@linux.alibaba.com/
- Patch #6: use SMCD_CLC_MAX_V2_GID_ENTRIES to indicate the max gid
  entries in CLC proposal and using SMC_MAX_V2_ISM_DEVS to indicate the
  max devices to propose;
- Patch #6: use i and i+1 in smc_find_ism_v2_device_serv();
- Patch #2: replace the large if-else block in smc_clc_send_confirm_accept()
  with 2 subfunctions;
- Fix missing byte order conversion of GID and token in CLC handshake,
  which is in a separate patch sending to net:
  https://lore.kernel.org/netdev/1701882157-87956-1-git-send-email-guwen@linux.alibaba.com/
- Patch #7: add extended GID in SMC-D lgr netlink attribute;

v3->v2:
https://lore.kernel.org/netdev/1701343695-122657-1-git-send-email-guwen@linux.alibaba.com/
- Rename smc_clc_fill_fce as smc_clc_fill_fce_v2x;
- Remove ISM_IDENT_MASK from drivers/s390/net/ism.h;
- Add explicitly assigning 'false' to ism_v2_capable in ism_dev_init();
- Remove smc_ism_set_v2_capable() helper for now, and introduce it in
  later loopback-ism implementation;

v2->v1:
- Fix sparse complaint;
- Rebase to the latest net-next;
====================

Signed-off-by: David S. Miller <davem@davemloft.net>
  • Loading branch information
davem330 committed Dec 26, 2023
2 parents e9301af + b3bf760 commit e3eb47f
Show file tree
Hide file tree
Showing 16 changed files with 448 additions and 295 deletions.
7 changes: 0 additions & 7 deletions drivers/s390/net/ism.h
Original file line number Diff line number Diff line change
Expand Up @@ -16,7 +16,6 @@
*/
#define ISM_DMB_WORD_OFFSET 1
#define ISM_DMB_BIT_OFFSET (ISM_DMB_WORD_OFFSET * 32)
#define ISM_IDENT_MASK 0x00FFFF

#define ISM_REG_SBA 0x1
#define ISM_REG_IEQ 0x2
Expand Down Expand Up @@ -192,12 +191,6 @@ struct ism_sba {
#define ISM_CREATE_REQ(dmb, idx, sf, offset) \
((dmb) | (idx) << 24 | (sf) << 23 | (offset))

struct ism_systemeid {
u8 seid_string[24];
u8 serial_number[4];
u8 type[4];
};

static inline void __ism_read_cmd(struct ism_dev *ism, void *data,
unsigned long offset, unsigned long len)
{
Expand Down
57 changes: 17 additions & 40 deletions drivers/s390/net/ism_drv.c
Original file line number Diff line number Diff line change
Expand Up @@ -36,6 +36,7 @@ static struct ism_client *clients[MAX_CLIENTS]; /* use an array rather than */
/* a list for fast mapping */
static u8 max_client;
static DEFINE_MUTEX(clients_lock);
static bool ism_v2_capable;
struct ism_dev_list {
struct list_head list;
struct mutex mutex; /* protects ism device list */
Expand Down Expand Up @@ -443,32 +444,6 @@ int ism_move(struct ism_dev *ism, u64 dmb_tok, unsigned int idx, bool sf,
}
EXPORT_SYMBOL_GPL(ism_move);

static struct ism_systemeid SYSTEM_EID = {
.seid_string = "IBM-SYSZ-ISMSEID00000000",
.serial_number = "0000",
.type = "0000",
};

static void ism_create_system_eid(void)
{
struct cpuid id;
u16 ident_tail;
char tmp[5];

get_cpu_id(&id);
ident_tail = (u16)(id.ident & ISM_IDENT_MASK);
snprintf(tmp, 5, "%04X", ident_tail);
memcpy(&SYSTEM_EID.serial_number, tmp, 4);
snprintf(tmp, 5, "%04X", id.machine);
memcpy(&SYSTEM_EID.type, tmp, 4);
}

u8 *ism_get_seid(void)
{
return SYSTEM_EID.seid_string;
}
EXPORT_SYMBOL_GPL(ism_get_seid);

static void ism_handle_event(struct ism_dev *ism)
{
struct ism_event *entry;
Expand Down Expand Up @@ -560,7 +535,9 @@ static int ism_dev_init(struct ism_dev *ism)

if (!ism_add_vlan_id(ism, ISM_RESERVED_VLANID))
/* hardware is V2 capable */
ism_create_system_eid();
ism_v2_capable = true;
else
ism_v2_capable = false;

mutex_lock(&ism_dev_list.mutex);
mutex_lock(&clients_lock);
Expand Down Expand Up @@ -665,8 +642,7 @@ static void ism_dev_exit(struct ism_dev *ism)
}
mutex_unlock(&clients_lock);

if (SYSTEM_EID.serial_number[0] != '0' ||
SYSTEM_EID.type[0] != '0')
if (ism_v2_capable)
ism_del_vlan_id(ism, ISM_RESERVED_VLANID);
unregister_ieq(ism);
unregister_sba(ism);
Expand Down Expand Up @@ -743,10 +719,10 @@ static int ism_query_rgid(struct ism_dev *ism, u64 rgid, u32 vid_valid,
return ism_cmd(ism, &cmd);
}

static int smcd_query_rgid(struct smcd_dev *smcd, u64 rgid, u32 vid_valid,
u32 vid)
static int smcd_query_rgid(struct smcd_dev *smcd, struct smcd_gid *rgid,
u32 vid_valid, u32 vid)
{
return ism_query_rgid(smcd->priv, rgid, vid_valid, vid);
return ism_query_rgid(smcd->priv, rgid->gid, vid_valid, vid);
}

static int smcd_register_dmb(struct smcd_dev *smcd, struct smcd_dmb *dmb,
Expand Down Expand Up @@ -797,10 +773,11 @@ static int ism_signal_ieq(struct ism_dev *ism, u64 rgid, u32 trigger_irq,
return ism_cmd(ism, &cmd);
}

static int smcd_signal_ieq(struct smcd_dev *smcd, u64 rgid, u32 trigger_irq,
u32 event_code, u64 info)
static int smcd_signal_ieq(struct smcd_dev *smcd, struct smcd_gid *rgid,
u32 trigger_irq, u32 event_code, u64 info)
{
return ism_signal_ieq(smcd->priv, rgid, trigger_irq, event_code, info);
return ism_signal_ieq(smcd->priv, rgid->gid,
trigger_irq, event_code, info);
}

static int smcd_move(struct smcd_dev *smcd, u64 dmb_tok, unsigned int idx,
Expand All @@ -812,18 +789,19 @@ static int smcd_move(struct smcd_dev *smcd, u64 dmb_tok, unsigned int idx,

static int smcd_supports_v2(void)
{
return SYSTEM_EID.serial_number[0] != '0' ||
SYSTEM_EID.type[0] != '0';
return ism_v2_capable;
}

static u64 ism_get_local_gid(struct ism_dev *ism)
{
return ism->local_gid;
}

static u64 smcd_get_local_gid(struct smcd_dev *smcd)
static void smcd_get_local_gid(struct smcd_dev *smcd,
struct smcd_gid *smcd_gid)
{
return ism_get_local_gid(smcd->priv);
smcd_gid->gid = ism_get_local_gid(smcd->priv);
smcd_gid->gid_ext = 0;
}

static u16 ism_get_chid(struct ism_dev *ism)
Expand Down Expand Up @@ -857,7 +835,6 @@ static const struct smcd_ops ism_ops = {
.signal_event = smcd_signal_ieq,
.move_data = smcd_move,
.supports_v2 = smcd_supports_v2,
.get_system_eid = ism_get_seid,
.get_local_gid = smcd_get_local_gid,
.get_chid = smcd_get_chid,
.get_dev = smcd_get_dev,
Expand Down
1 change: 0 additions & 1 deletion include/linux/ism.h
Original file line number Diff line number Diff line change
Expand Up @@ -86,7 +86,6 @@ int ism_register_dmb(struct ism_dev *dev, struct ism_dmb *dmb,
int ism_unregister_dmb(struct ism_dev *dev, struct ism_dmb *dmb);
int ism_move(struct ism_dev *dev, u64 dmb_tok, unsigned int idx, bool sf,
unsigned int offset, void *data, unsigned int size);
u8 *ism_get_seid(void);

const struct smcd_ops *ism_get_smcd_ops(void);

Expand Down
16 changes: 10 additions & 6 deletions include/net/smc.h
Original file line number Diff line number Diff line change
Expand Up @@ -52,24 +52,28 @@ struct smcd_dmb {
struct smcd_dev;
struct ism_client;

struct smcd_gid {
u64 gid;
u64 gid_ext;
};

struct smcd_ops {
int (*query_remote_gid)(struct smcd_dev *dev, u64 rgid, u32 vid_valid,
u32 vid);
int (*query_remote_gid)(struct smcd_dev *dev, struct smcd_gid *rgid,
u32 vid_valid, u32 vid);
int (*register_dmb)(struct smcd_dev *dev, struct smcd_dmb *dmb,
struct ism_client *client);
int (*unregister_dmb)(struct smcd_dev *dev, struct smcd_dmb *dmb);
int (*add_vlan_id)(struct smcd_dev *dev, u64 vlan_id);
int (*del_vlan_id)(struct smcd_dev *dev, u64 vlan_id);
int (*set_vlan_required)(struct smcd_dev *dev);
int (*reset_vlan_required)(struct smcd_dev *dev);
int (*signal_event)(struct smcd_dev *dev, u64 rgid, u32 trigger_irq,
u32 event_code, u64 info);
int (*signal_event)(struct smcd_dev *dev, struct smcd_gid *rgid,
u32 trigger_irq, u32 event_code, u64 info);
int (*move_data)(struct smcd_dev *dev, u64 dmb_tok, unsigned int idx,
bool sf, unsigned int offset, void *data,
unsigned int size);
int (*supports_v2)(void);
u8* (*get_system_eid)(void);
u64 (*get_local_gid)(struct smcd_dev *dev);
void (*get_local_gid)(struct smcd_dev *dev, struct smcd_gid *gid);
u16 (*get_chid)(struct smcd_dev *dev);
struct device* (*get_dev)(struct smcd_dev *dev);
};
Expand Down
2 changes: 2 additions & 0 deletions include/uapi/linux/smc.h
Original file line number Diff line number Diff line change
Expand Up @@ -160,6 +160,8 @@ enum {
SMC_NLA_LGR_D_CHID, /* u16 */
SMC_NLA_LGR_D_PAD, /* flag */
SMC_NLA_LGR_D_V2_COMMON, /* nest */
SMC_NLA_LGR_D_EXT_GID, /* u64 */
SMC_NLA_LGR_D_PEER_EXT_GID, /* u64 */
__SMC_NLA_LGR_D_MAX,
SMC_NLA_LGR_D_MAX = __SMC_NLA_LGR_D_MAX - 1
};
Expand Down
2 changes: 2 additions & 0 deletions include/uapi/linux/smc_diag.h
Original file line number Diff line number Diff line change
Expand Up @@ -107,6 +107,8 @@ struct smcd_diag_dmbinfo { /* SMC-D Socket internals */
__aligned_u64 my_gid; /* My GID */
__aligned_u64 token; /* Token of DMB */
__aligned_u64 peer_token; /* Token of remote DMBE */
__aligned_u64 peer_gid_ext; /* Peer GID (extended part) */
__aligned_u64 my_gid_ext; /* My GID (extended part) */
};

#endif /* _UAPI_SMC_DIAG_H_ */
Loading

0 comments on commit e3eb47f

Please sign in to comment.