I/O Request Handling
This section discusses I/O request processing in detail.
User Addresses
When a user thread issues a write(2) system call, the thread passes the address
of a buffer in user space:
char buffer[] = "python";
count = write(fd, buffer, strlen(buffer) + 1);
The system builds a uio(9S) structure to describe this transfer by allocating an
iovec(9S) structure and setting the iov_base field to the address passed to
write(2), in this case, buffer. The uio(9S) structure is passed to the driver
write(9E) routine. See Vectored I/O for more information about the uio(9S) structure.
The address in the iovec(9S) is in user space, not kernel space. Thus,
the address is neither guaranteed to be currently in memory nor to
be a valid address. In either case, accessing a user address directly from
the device driver or from the kernel could crash the system. Thus, device
drivers should never access user addresses directly. Instead, a data transfer routine in
the Solaris DDI/DKI should be used to transfer data into or out of
the kernel. These routines can handle page faults. The DDI/DKI routines can bring
in the proper user page to continue the copy transparently. Alternatively, the routines
can return an error on an invalid access.
copyout(9F) can be used to copy data from kernel space to user space.
copyin(9F) can copy data from user space to kernel space. ddi_copyout(9F) and ddi_copyin(9F)
operate similarly but are to be used in the ioctl(9E) routine. copyin(9F) and
copyout(9F) can be used on the buffer described by each iovec(9S) structure, or
uiomove(9F) can perform the entire transfer to or from a contiguous area of
driver or device memory.
Vectored I/O
In character drivers, transfers are described by a uio(9S) structure. The uio(9S)
structure contains information about the direction and size of the transfer, plus an
array of buffers for one end of the transfer. The other end is
the device.
The uio(9S) structure contains the following members:
iovec_t *uio_iov; /* base address of the iovec */
/* buffer description array */
int uio_iovcnt; /* the number of iovec structures */
off_t uio_offset; /* 32-bit offset into file where */
/* data is transferred from or to */
offset_t uio_loffset; /* 64-bit offset into file where */
/* data is transferred from or to */
uio_seg_t uio_segflg; /* identifies the type of I/O transfer */
/* UIO_SYSSPACE: kernel <-> kernel */
/* UIO_USERSPACE: kernel <-> user */
short uio_fmode; /* file mode flags (not driver setTable) */
daddr_t uio_limit; /* 32-bit ulimit for file (maximum */
/* block offset). not driver settable. */
diskaddr_t uio_llimit; /* 64-bit ulimit for file (maximum block */
/* block offset). not driver settable. */
int uio_resid; /* amount (in bytes) not */
/* transferred on completion */A uio(9S) structure is passed to the driver read(9E) and write(9E) entry points.
This structure is generalized to support what is called gather-write and scatter-read. When
writing to a device, the data buffers to be written do not have
to be contiguous in application memory. Similarly, data that is transferred from a
device into memory comes off in a contiguous stream but can go into
noncontiguous areas of application memory. See the readv(2), writev(2), pread(2), and pwrite(2)
man pages for more information on scatter-gather I/O.
Each buffer is described by an iovec(9S) structure. This structure contains a pointer
to the data area and the number of bytes to be transferred.
caddr_t iov_base; /* address of buffer */
int iov_len; /* amount to transfer */
The uio structure contains a pointer to an array of iovec(9S) structures. The base
address of this array is held in uio_iov, and the number of elements
is stored in uio_iovcnt.
The uio_offset field contains the 32-bit offset into the device at which the
application needs to begin the transfer. uio_loffset is used for 64-bit file
offsets. If the device does not support the notion of an offset, these
fields can be safely ignored. The driver should interpret either uio_offset or uio_loffset,
but not both. If the driver has set the D_64BIT flag in
the cb_ops(9S) structure, that driver should use uio_loffset.
The uio_resid field starts out as the number of bytes to be transferred,
that is, the sum of all the iov_len fields in uio_iov. This field
must be set by the driver to the number of bytes that were
not transferred before returning. The read(2) and write(2) system calls use the return value
from the read(9E) and write(9E) entry points to determine failed transfers. If a
failure occurs, these routines return -1. If the return value indicates success, the
system calls return the number of bytes requested minus uio_resid. If uio_resid is not
changed by the driver, the read(2) and write(2) calls return 0. A
return value of 0 indicates end-of-file, even though all the data has been
transferred.
The support routines uiomove(9F), physio(9F), and aphysio(9F) update the uio(9S) structure
directly. These support routines update the device offset to account for the data
transfer. Neither the uio_offset or uio_loffset fields need to be adjusted when the driver
is used with a seekable device that uses the concept of position. I/O
performed to a device in this manner is constrained by the maximum possible
value of uio_offset or uio_loffset. An example of such a usage is
raw I/O on a disk.
If the device has no concept of position, the driver can take
the following steps:
Save uio_offset or uio_loffset.
Perform the I/O operation.
Restore uio_offset or uio_loffset to the field's initial value.
I/O that is performed to a device in this manner is not
constrained by the maximum possible value of uio_offset or uio_loffset. An example of
this type of usage is I/O on a serial line.
The following example shows one way to preserve uio_loffset in the read(9E)
function.
static int
xxread(dev_t dev, struct uio *uio_p, cred_t *cred_p)
{
offset_t off;
/* ... */
off = uio_p->uio_loffset; /* save the offset */
/* do the transfer */
uio_p->uio_loffset = off; /* restore it */
}
Differences Between Synchronous and Asynchronous I/O
Data transfers can be synchronous or asynchronous. The determining factor is whether the
entry point that schedules the transfer returns immediately or waits until the I/O
has been completed.
The read(9E) and write(9E) entry points are synchronous entry points. The transfer must
not return until the I/O is complete. Upon return from the routines, the
process knows whether the transfer has succeeded.
The aread(9E) and awrite(9E) entry points are asynchronous entry points. Asynchronous entry
points schedule the I/O and return immediately. Upon return, the process that issues
the request knows that the I/O is scheduled and that the status of
the I/O must be determined later. In the meantime, the process can perform
other operations.
With an asynchronous I/O request to the kernel, the process is not required
to wait while the I/O is in process. A process can perform multiple
I/O requests and allow the kernel to handle the data transfer details. Asynchronous
I/O requests enable applications such as transaction processing to use concurrent programming methods
to increase performance or response time. Any performance boost for applications that use asynchronous
I/O, however, comes at the expense of greater programming complexity.
Data Transfer Methods
Data can be transferred using either programmed I/O or DMA. These data transfer
methods can be used either by synchronous or by asynchronous entry points, depending
on the capabilities of the device.
Programmed I/O Transfers
Programmed I/O devices rely on the CPU to perform the data transfer. Programmed
I/O data transfers are identical to other read and write operations for device
registers. Various data access routines are used to read or store values to
device memory.
uiomove(9F) can be used to transfer data to some programmed I/O devices.
uiomove(9F) transfers data between the user space, as defined by the uio(9S) structure,
and the kernel. uiomove() can handle page faults, so the memory to
which data is transferred need not be locked down. uiomove() also updates the
uio_resid field in the uio(9S) structure. The following example shows one way to
write a ramdisk read(9E) routine. It uses synchronous I/O and relies on the
presence of the following fields in the ramdisk state structure:
caddr_t ram; /* base address of ramdisk */
int ramsize; /* size of the ramdisk */
Example 15-3 Ramdisk read(9E) Routine Using uiomove(9F)
static int
rd_read(dev_t dev, struct uio *uiop, cred_t *credp)
{
rd_devstate_t *rsp;
rsp = ddi_get_soft_state(rd_statep, getminor(dev));
if (rsp == NULL)
return (ENXIO);
if (uiop->uio_offset >= rsp->ramsize)
return (EINVAL);
/*
* uiomove takes the offset into the kernel buffer,
* the data transfer count (minimum of the requested and
* the remaining data), the UIO_READ flag, and a pointer
* to the uio structure.
*/
return (uiomove(rsp->ram + uiop->uio_offset,
min(uiop->uio_resid, rsp->ramsize - uiop->uio_offset),
UIO_READ, uiop));
}Another example of programmed I/O would be a driver that writes data one
byte at a time directly to the device's memory. Each byte is
retrieved from the uio(9S) structure by using uwritec(9F). The byte is then sent to the
device. read(9E) can use ureadc(9F) to transfer a byte from the device to
the area described by the uio(9S) structure.
Example 15-4 Programmed I/O write(9E) Routine Using uwritec(9F)
static int
xxwrite(dev_t dev, struct uio *uiop, cred_t *credp)
{
int value;
struct xxstate *xsp;
xsp = ddi_get_soft_state(statep, getminor(dev));
if (xsp == NULL)
return (ENXIO);
/* if the device implements a power manageable component, do this: */
pm_busy_component(xsp->dip, 0);
if (xsp->pm_suspended)
pm_raise_power(xsp->dip, normal power);
while (uiop->uio_resid > 0) {
/*
* do the programmed I/O access
*/
value = uwritec(uiop);
if (value == -1)
return (EFAULT);
ddi_put8(xsp->data_access_handle, &xsp->regp->data,
(uint8_t)value);
ddi_put8(xsp->data_access_handle, &xsp->regp->csr,
START_TRANSFER);
/*
* this device requires a ten microsecond delay
* between writes
*/
drv_usecwait(10);
}
pm_idle_component(xsp->dip, 0);
return (0);
}
DMA Transfers (Synchronous)
Character drivers generally use physio(9F) to do the setup work for DMA transfers
in read(9E) and write(9E), as is shown in Example 15-5.
int physio(int (*strat)(struct buf *), struct buf *bp,
dev_t dev, int rw, void (*mincnt)(struct buf *),
struct uio *uio);physio(9F) requires the driver to provide the address of a strategy(9E) routine.
physio(9F) ensures that memory space is locked down, that is, memory cannot be
paged out, for the duration of the data transfer. This lock-down is necessary
for DMA transfers because DMA transfers cannot handle page faults. physio(9F) also
provides an automated way of breaking a larger transfer into a series of
smaller, more manageable ones. See minphys() Entry Point for more information.
Example 15-5 read(9E) and write(9E) Routines Using physio(9F)
static int
xxread(dev_t dev, struct uio *uiop, cred_t *credp)
{
struct xxstate *xsp;
int ret;
xsp = ddi_get_soft_state(statep, getminor(dev));
if (xsp == NULL)
return (ENXIO);
ret = physio(xxstrategy, NULL, dev, B_READ, xxminphys, uiop);
return (ret);
}
static int
xxwrite(dev_t dev, struct uio *uiop, cred_t *credp)
{
struct xxstate *xsp;
int ret;
xsp = ddi_get_soft_state(statep, getminor(dev));
if (xsp == NULL)
return (ENXIO);
ret = physio(xxstrategy, NULL, dev, B_WRITE, xxminphys, uiop);
return (ret);
}In the call to physio(9F), xxstrategy is a pointer to the driver
strategy() routine. Passing NULL as the buf(9S) structure pointer tells physio(9F) to allocate
a buf(9S) structure. If the driver must provide physio(9F) with a buf(9S) structure, getrbuf(9F)
should be used to allocate the structure. physio(9F) returns zero if the transfer completes
successfully, or an error number on failure. After calling strategy(9E), physio(9F) calls
biowait(9F) to block until the transfer either completes or fails. The return
value of physio(9F) is determined by the error field in the buf(9S) structure set
by bioerror(9F).
DMA Transfers (Asynchronous)
Character drivers that support aread(9E) and awrite(9E) use aphysio(9F) instead of
physio(9F).
int aphysio(int (*strat)(struct buf *), int (*cancel)(struct buf *),
dev_t dev, int rw, void (*mincnt)(struct buf *),
struct aio_req *aio_reqp);
Note - The address of anocancel(9F) is the only value that can currently be
passed as the second argument to aphysio(9F).
aphysio(9F) requires the driver to pass the address of a strategy(9E) routine.
aphysio(9F) ensures that memory space is locked down, that is, cannot be paged
out, for the duration of the data transfer. This lock-down is necessary for
DMA transfers because DMA transfers cannot handle page faults. aphysio(9F) also provides
an automated way of breaking a larger transfer into a series of smaller,
more manageable ones. See minphys() Entry Point for more information.
Example 15-5 and Example 15-6 demonstrate that the aread(9E) and awrite(9E) entry points differ only
slightly from the read(9E) and write(9E) entry points. The difference is primarily
in their use of aphysio(9F) instead of physio(9F).
Example 15-6 aread(9E) and awrite(9E) Routines Using aphysio(9F)
static int
xxaread(dev_t dev, struct aio_req *aiop, cred_t *cred_p)
{
struct xxstate *xsp;
xsp = ddi_get_soft_state(statep, getminor(dev));
if (xsp == NULL)
return (ENXIO);
return (aphysio(xxstrategy, anocancel, dev, B_READ,
xxminphys, aiop));
}
static int
xxawrite(dev_t dev, struct aio_req *aiop, cred_t *cred_p)
{
struct xxstate *xsp;
xsp = ddi_get_soft_state(statep, getminor(dev));
if (xsp == NULL)
return (ENXIO);
return (aphysio(xxstrategy, anocancel, dev, B_WRITE,
xxminphys,aiop));
}In the call to aphysio(9F), xxstrategy() is a pointer to the
driver strategy routine. aiop is a pointer to the aio_req(9S) structure. aiop
is passed to aread(9E) and awrite(9E). aio_req(9S) describes where the data
is to be stored in user space. aphysio(9F) returns zero if
the I/O request is scheduled successfully or an error number on failure. After
calling strategy(9E), aphysio(9F) returns without waiting for the I/O to complete or
fail.
minphys() Entry Point
The minphys() entry point is a pointer to a function to be called
by physio(9F) or aphysio(9F). The purpose of xxminphys is to ensure that
the size of the requested transfer does not exceed a driver-imposed limit. If
the user requests a larger transfer, strategy(9E) is called repeatedly, requesting no more
than the imposed limit at a time. This approach is important because DMA
resources are limited. Drivers for slow devices, such as printers, should be careful
not to tie up resources for a long time.
Usually, a driver passes the address of the kernel function minphys(9F),
but the driver can define its own xxminphys() routine instead. The job of xxminphys()
is to keep the b_bcount field of the buf(9S) structure under a driver's limit.
The driver should adhere to other system limits as well. For example, the
driver's xxminphys() routine should call the system minphys(9F) routine after setting the b_bcount field
and before returning.
Example 15-7 minphys(9F) Routine
#define XXMINVAL (512 << 10) /* 512 KB */
static void
xxminphys(struct buf *bp)
{
if (bp->b_bcount > XXMINVAL)
bp->b_bcount = XXMINVAL
minphys(bp);
}
strategy() Entry Point
The strategy(9E) routine originated in block drivers. The strategy function got its
name from implementing a strategy for efficient queuing of I/O requests to a
block device. A driver for a character-oriented device can also use a strategy(9E)
routine. In the character I/O model presented here, strategy(9E) does not maintain
a queue of requests, but rather services one request at a time.
In the following example, the strategy(9E) routine for a character-oriented DMA device allocates
DMA resources for synchronous data transfer. strategy() starts the command by programming
the device register. See Chapter 9, Direct Memory Access (DMA) for a detailed description.
Note - strategy(9E) does not receive a device number (dev_t) as a parameter. Instead, the device
number is retrieved from the b_edev field of the buf(9S) structure passed to strategy(9E).
Example 15-8 strategy(9E) Routine
static int
xxstrategy(struct buf *bp)
{
minor_t instance;
struct xxstate *xsp;
ddi_dma_cookie_t cookie;
instance = getminor(bp->b_edev);
xsp = ddi_get_soft_state(statep, instance);
/* ... */
* If the device has power manageable components,
* mark the device busy with pm_busy_components(9F),
* and then ensure that the device is
* powered up by calling pm_raise_power(9F).
*/
/* Set up DMA resources with ddi_dma_alloc_handle(9F) and
* ddi_dma_buf_bind_handle(9F).
*/
xsp->bp = bp; /* remember bp */
/* Program DMA engine and start command */
return (0);
}
Note - Although strategy() is declared to return an int, strategy() must always return
zero.
On completion of the DMA transfer, the device generates an interrupt, causing the
interrupt routine to be called. In the following example, xxintr() receives a
pointer to the state structure for the device that might have generated the
interrupt.
Example 15-9 Interrupt Routine
static u_int
xxintr(caddr_t arg)
{
struct xxstate *xsp = (struct xxstate *)arg;
if ( /* device did not interrupt */ ) {
return (DDI_INTR_UNCLAIMED);
}
if ( /* error */ ) {
/* error handling */
}
/* Release any resources used in the transfer, such as DMA resources.
* ddi_dma_unbind_handle(9F) and ddi_dma_free_handle(9F)
* Notify threads that the transfer is complete.
*/
biodone(xsp->bp);
return (DDI_INTR_CLAIMED);
}The driver indicates an error by calling bioerror(9F). The driver must call
biodone(9F) when the transfer is complete or after indicating an error with
bioerror(9F).
