On 11/27/2015 7:44 PM, Matthew Grooms wrote:
I spent the day looking over the FreeBSD cam and scsi_da source code.
After sprinkling a bunch of printf's around to see what code paths
were being called, It's obvious that Edward was correct in assuming
that ESXi doesn't return any 'Unit Attention' sense information in
response to a 'Read Capacity' request. This kinda makes sense as ESXi
emulates SCSI-2 disk devices and, as far as I can tell, the 0x2A/0x09
ASC/ASCQ sense code that denotes 'Capacity Data Has Changed' wasn't
defined until the SCSCI-3 spec. It's frustrating that the only way to
get the scsci_da code to call reprobe() is by receiving a command from
the device. Would something like this work? ...
1) Register a callback using xpt_register_async( daasync,
AC_REPROBE_DEVICE, path ) that calls reprobe()
2) Implement a new IOCTL in cam_xpt that camcontrol can call with the
bus:target:lun as the argument
3) have cam_xpt capture the IOCTL request and call xpt_async(
AC_REPROBE_DEVICE, path ) as a result
This way users would have the option of manually asking cam to
communicate the new size to geom. The only option now is one or more
reboots to gain access to the increased disk capacity. If this sounds
like a reasonable approach, I'll take a stab at implementing it.
Here is a proof of concept patch. I'm a complete noob when it comes to
cam, scsi or freebsd kernel development for that matter, so I'm sure it
could have been done a better way. In any case, I added a new command to
camcontrol that allows you to specify a bus, target and lun as an
argument. For example ...
# camcontrol readcap da1 -h
Device Size: 32 G, Block Length: 512 bytes
# gpart show da1
=> 40 58720176 da1 GPT (28G)
40 58720176 1 freebsd-ufs (28G)
Note, I resized the VMDK disk in ESXi. The camcontrol output shows the
size as 32G but geom thinks its 28G.
# camcontrol devlist
<NECVMWar VMware IDE CDR10 1.00> at scbus1 target 0 lun 0 (cd0,pass0)
<VMware Virtual disk 1.0> at scbus2 target 0 lun 0 (pass1,da0)
<VMware Virtual disk 1.0> at scbus2 target 1 lun 0 (pass2,da1)
<FREEBSD CTLDISK 0001> at scbus3 target 0 lun 0 (da2,pass3)
# camcontrol reprobe 2:1:0
This generates an event that is captured by the scsci da device to
forces a reprobe. The kernel output looks almost identical to when the
'Unit Attention' sense data is received ...
Nov 28 17:46:13 iscsi-i kernel: (da1:mpt0:0:1:0): Re-probe requested
Nov 28 17:46:13 iscsi-i kernel: GEOM_PART: da1 was automatically resized.
Nov 28 17:46:13 iscsi-i kernel: Use `gpart commit da1` to save changes
or `gpart undo da1` to revert them.
Now that geom knows about the increased disk capacity, I can increase
the partition size and grow the fs ...
[root@iscsi-i /home/mgrooms]# gpart show da1
=> 40 67108784 da1 GPT (32G)
40 58720176 1 freebsd-ufs (28G)
58720216 8388608 - free - (4.0G)
# gpart resize -i 1 da1
da1p1 resized
# growfs da1p1
Device is mounted read-write; resizing will result in temporary write
suspension for /var/data1.
It's strongly recommended to make a backup before growing the file system.
OK to grow filesystem on /dev/da1p1, mounted on /var/data1, from 28GB to
32GB? [Yes/No] Yes
super-block backups (for fsck_ffs -b #) at:
58983232, 60265472, 61547712, 62829952, 64112192, 65394432, 66676672
# df -h
Filesystem Size Used Avail Capacity Mounted on
/dev/da0p3 18G 5.3G 12G 31% /
devfs 1.0K 1.0K 0B 100% /dev
/dev/da1p1 31G 32M 28G 0% /var/data1
/dev/da2p1 15G 32M 14G 0% /var/data2
Sure would be nice to have something like this in the tree. It's really
a drag to have to reboot production VMs to increase disk capacity when
it could be easily avoided. I'm not sure what the correct IOCTL should
look like. Maybe CAMIOCOMMAND is a better way to go? If someone with
some experience with the cam/scsi subsystems was willing to give me some
direction I'd be willing to try and rewrite the patch in a way that
would be commit worthy. I just need some direction.
Thanks,
-Matthew
Index: lib/libcam/camlib.c
===================================================================
--- lib/libcam/camlib.c (revision 291390)
+++ lib/libcam/camlib.c (working copy)
@@ -752,3 +752,41 @@
bcopy(src, dst, sizeof(struct cam_device));
}
+
+/*
+ * Send a reprobe unit request for a given bus, target and lun
+ */
+int
+cam_reprobe_btl(path_id_t path_id, target_id_t target_id, lun_id_t target_lun)
+{
+ int fd;
+ char *func_name = "cam_reprobe_btl";
+ union ccb ccb;
+
+ if ((fd = open(XPT_DEVICE, O_RDWR)) < 0) {
+ snprintf(cam_errbuf, CAM_ERRBUF_SIZE,
+ "%s: couldn't open %s\n%s: %s", func_name, XPT_DEVICE,
+ func_name, strerror(errno));
+ return(-1);
+ }
+
+ /* Setup our request ccb */
+ bzero(&ccb.ccb_h, sizeof(struct ccb_hdr));
+ ccb.ccb_h.path_id = path_id;
+ ccb.ccb_h.target_id = target_id;
+ ccb.ccb_h.target_lun = target_lun;
+
+ /*
+ * Attempt to issue a unit reprobe request. This ioctl will fail if the
+ * kernel can't find a unit for the given path, target and lun.
+ */
+ if (ioctl(fd, CAMUNITREPROBE, &ccb) != 0) {
+ snprintf(cam_errbuf, CAM_ERRBUF_SIZE,
+ "%s: failed to issue unit reprobe request", func_name
);
+ close(fd);
+ return(-1);
+ }
+
+ close(fd);
+ return 0;
+}
Index: lib/libcam/camlib.h
===================================================================
--- lib/libcam/camlib.h (revision 291390)
+++ lib/libcam/camlib.h (working copy)
@@ -146,6 +146,8 @@
struct cam_device *dst);
int cam_get_device(const char *path, char *dev_name,
int devnamelen, int *unit);
+int cam_reprobe_btl(path_id_t path_id, target_id_t
target_id,
+ lun_id_t target_lun);
/*
* Buffer encoding/decoding routines, from the old SCSI library.
Index: sbin/camcontrol/camcontrol.c
===================================================================
--- sbin/camcontrol/camcontrol.c (revision 291390)
+++ sbin/camcontrol/camcontrol.c (working copy)
@@ -100,7 +100,8 @@
CAM_CMD_APM = 0x00000021,
CAM_CMD_AAM = 0x00000022,
CAM_CMD_ATTRIB = 0x00000023,
- CAM_CMD_OPCODES = 0x00000024
+ CAM_CMD_OPCODES = 0x00000024,
+ CAM_CMD_REPROBE = 0x00000025
} cam_cmdmask;
typedef enum {
@@ -193,6 +194,7 @@
#endif /* MINIMALISTIC */
{"rescan", CAM_CMD_RESCAN, CAM_ARG_NONE, NULL},
{"reset", CAM_CMD_RESET, CAM_ARG_NONE, NULL},
+ {"reprobe", CAM_CMD_REPROBE, CAM_ARG_NONE, NULL},
#ifndef MINIMALISTIC
{"cmd", CAM_CMD_SCSI_CMD, CAM_ARG_NONE, scsicmd_opts},
{"command", CAM_CMD_SCSI_CMD, CAM_ARG_NONE, scsicmd_opts},
@@ -3127,6 +3129,34 @@
}
static int
+doreprobe(int argc, char **argv)
+{
+ static const char must[] =
+ "you must specify a bus:target:lun to reprobe";
+ int rv;
+ path_id_t bus = CAM_BUS_WILDCARD;
+ target_id_t target = CAM_TARGET_WILDCARD;
+ lun_id_t lun = CAM_LUN_WILDCARD;
+ char *tstr;
+
+ if (argc < 3) {
+ warnx(must);
+ return(1);
+ }
+
+ tstr = argv[optind];
+ while (isspace(*tstr) && (*tstr != '\0'))
+ tstr++;
+ rv = parse_btl(argv[optind], &bus, &target, &lun, &arglist);
+ if (rv != 3) {
+ warnx(must);
+ return(1);
+ }
+
+ return(cam_reprobe_btl(bus, target, lun));
+}
+
+static int
rescan_or_reset_bus(path_id_t bus, int rescan)
{
union ccb ccb, matchccb;
@@ -8685,6 +8715,7 @@
#endif /* MINIMALISTIC */
" camcontrol rescan <all | bus[:target:lun]>\n"
" camcontrol reset <all | bus[:target:lun]>\n"
+" camcontrol reprobe bus:target:lun\n"
#ifndef MINIMALISTIC
" camcontrol defects [dev_id][generic args] <-f format> [-P][-G]\n"
" [-q][-s][-S offset][-X]\n"
@@ -9053,6 +9084,7 @@
*/
if ((cmdlist == CAM_CMD_RESCAN)
|| (cmdlist == CAM_CMD_RESET)
+ || (cmdlist == CAM_CMD_REPROBE)
|| (cmdlist == CAM_CMD_DEVTREE)
|| (cmdlist == CAM_CMD_USAGE)
|| (cmdlist == CAM_CMD_DEBUG))
@@ -9204,6 +9236,9 @@
case CAM_CMD_RESET:
error = dorescan_or_reset(argc, argv, 0);
break;
+ case CAM_CMD_REPROBE:
+ error = doreprobe(argc, argv);
+ break;
#ifndef MINIMALISTIC
case CAM_CMD_READ_DEFECTS:
error = readdefects(cam_dev, argc, argv, combinedopt,
Index: sys/amd64/conf/GENERIC
===================================================================
--- sys/amd64/conf/GENERIC (revision 291390)
+++ sys/amd64/conf/GENERIC (working copy)
@@ -366,3 +366,6 @@
# The crypto framework is required by IPSEC
device crypto # Required by IPSEC
+
+# Cam Debug Support
+options CAMDEBUG
Index: sys/cam/cam_ccb.h
===================================================================
--- sys/cam/cam_ccb.h (revision 291390)
+++ sys/cam/cam_ccb.h (working copy)
@@ -767,6 +767,7 @@
* Definitions for the asynchronous callback CCB fields.
*/
typedef enum {
+ AC_UNIT_REPROBE = 0x8000,/* Device reprobe user request */
AC_UNIT_ATTENTION = 0x4000,/* Device reported UNIT ATTENTION */
AC_ADVINFO_CHANGED = 0x2000,/* Advance info might have changes */
AC_CONTRACT = 0x1000,/* A contractual callback */
Index: sys/cam/cam_xpt.c
===================================================================
--- sys/cam/cam_xpt.c (revision 291390)
+++ sys/cam/cam_xpt.c (working copy)
@@ -717,7 +717,41 @@
}
xpt_unlock_buses();
break;
+ }
+ case CAMUNITREPROBE: {
+ union ccb *inccb;
+ struct cam_path path;
+
+ inccb = (union ccb *)addr;
+
+ /*
+ * Make sure a specific bus, target and lun were specified.
+ */
+ if (inccb->ccb_h.path_id == CAM_BUS_WILDCARD ||
+ inccb->ccb_h.target_id == CAM_TARGET_WILDCARD ||
+ inccb->ccb_h.target_lun == CAM_LUN_WILDCARD) {
+ error = EINVAL;
+ break;
}
+
+ /*
+ * Compile a path using the bus, target, and lun the user
passed in.
+ */
+ if (xpt_compile_path(&path, NULL,
+ inccb->ccb_h.path_id,
+ inccb->ccb_h.target_id,
+ inccb->ccb_h.target_lun) !=
+ CAM_REQ_CMP){
+ error = EINVAL;
+ break;
+ }
+
+ /*
+ * Notify interested parties
+ */
+ xpt_async(AC_UNIT_REPROBE, &path, NULL);
+ break;
+ }
default:
error = ENOTTY;
break;
Index: sys/cam/scsi/scsi_da.c
===================================================================
--- sys/cam/scsi/scsi_da.c (revision 291390)
+++ sys/cam/scsi/scsi_da.c (working copy)
@@ -1702,6 +1702,17 @@
}
break;
}
+ case AC_UNIT_REPROBE:
+ {
+ softc = (struct da_softc *)periph->softc;
+
+ xpt_print(periph->path,
+ "Re-probe requested\n");
+ softc->flags &= ~DA_FLAG_PROBED;
+ dareprobe(periph);
+
+ break;
+ }
case AC_UNIT_ATTENTION:
{
union ccb *ccb;
@@ -2240,7 +2251,7 @@
* would be to not attach the device on failure.
*/
xpt_register_async(AC_SENT_BDR | AC_BUS_RESET | AC_LOST_DEVICE |
- AC_ADVINFO_CHANGED | AC_SCSI_AEN | AC_UNIT_ATTENTION,
+ AC_ADVINFO_CHANGED | AC_SCSI_AEN | AC_UNIT_ATTENTION |
AC_UNIT_REPROBE,
daasync, periph, periph->path);
/*
@@ -3244,12 +3255,11 @@
if (have_sense)
scsi_sense_print(
&done_ccb->csio);
- else {
+ else
xpt_print(periph->path,
"got CAM status %#x\n",
done_ccb->ccb_h.status);
- }
-
+
xpt_print(periph->path, "fatal error, "
"failed to attach to device\n");
Index: sys/cam/scsi/scsi_pass.h
===================================================================
--- sys/cam/scsi/scsi_pass.h (revision 291390)
+++ sys/cam/scsi/scsi_pass.h (working copy)
@@ -38,5 +38,6 @@
*/
#define CAMIOCOMMAND _IOWR(CAM_VERSION, 2, union ccb)
#define CAMGETPASSTHRU _IOWR(CAM_VERSION, 3, union ccb)
+#define CAMUNITREPROBE _IOWR(CAM_VERSION, 4, union ccb)
#endif
_______________________________________________
freebsd-current@freebsd.org mailing list
https://lists.freebsd.org/mailman/listinfo/freebsd-current
To unsubscribe, send any mail to "freebsd-current-unsubscr...@freebsd.org"
