7.2.1. Optimizing Queries with EXPLAIN
EXPLAIN tbl_name
Or:
EXPLAIN [EXTENDED | PARTITIONS] SELECT select_options
The EXPLAIN
statement can be used either as a
synonym for DESCRIBE
or as a way to obtain
information about how MySQL executes a SELECT
statement:
EXPLAIN
tbl_name
is synonymous
with DESCRIBE
tbl_name
or
SHOW COLUMNS FROM
tbl_name
.
When you precede a SELECT
statement with
the keyword EXPLAIN
, MySQL displays
information from the optimizer about the query execution
plan. That is, MySQL explains how it would process the
SELECT
, including information about how
tables are joined and in which order.
EXPLAIN PARTITIONS
is available beginning
with MySQL 5.1.5. It is useful only when examining queries
involving partitioned tables. For details, see
Section 17.3.4, “Obtaining Information About Partitions”.
This section describes the second use of
EXPLAIN
for obtaining query execution plan
information. For a description of the
DESCRIBE
and SHOW COLUMNS
statements, see Section 13.3.1, “DESCRIBE
Syntax”, and
Section 13.5.4.3, “SHOW COLUMNS
Syntax”.
With the help of EXPLAIN
, you can see where
you should add indexes to tables to get a faster
SELECT
that uses indexes to find rows. You
can also use EXPLAIN
to check whether the
optimizer joins the tables in an optimal order. To force the
optimizer to use a join order corresponding to the order in
which the tables are named in the SELECT
statement, begin the statement with SELECT
STRAIGHT_JOIN
rather than just
SELECT
.
If you have a problem with indexes not being used when you
believe that they should be, you should run ANALYZE
TABLE
to update table statistics such as cardinality
of keys, that can affect the choices the optimizer makes. See
Section 13.5.2.1, “ANALYZE TABLE
Syntax”.
EXPLAIN
returns a row of information for each
table used in the SELECT
statement. The
tables are listed in the output in the order that MySQL would
read them while processing the query. MySQL resolves all joins
using a single-sweep multi-join method.
This means that MySQL reads a row from the first table, and then
finds a matching row in the second table, the third table, and
so on. When all tables are processed, MySQL outputs the selected
columns and backtracks through the table list until a table is
found for which there are more matching rows. The next row is
read from this table and the process continues with the next
table.
When the EXTENDED
keyword is used,
EXPLAIN
produces extra information that can
be viewed by issuing a SHOW WARNINGS
statement following the EXPLAIN
statement.
This information displays how the optimizer qualifies table and
column names in the SELECT
statement, what
the SELECT
looks like after the application
of rewriting and optimization rules, and possibly other notes
about the optimization process.
Note: You cannot use the
EXTENDED
and PARTITIONS
keywords together in the same EXPLAIN
statement.
Each output row from EXPLAIN
provides
information about one table, and each row contains the following
columns:
-
id
The SELECT
identifier. This is the
sequential number of the SELECT
within
the query.
-
select_type
The type of SELECT
, which can be any of
those shown in the following table:
DEPENDENT
typically signifies the use of
a correlated subquery. See
Section 13.2.8.7, “Correlated Subqueries”.
-
table
The table to which the row of output refers.
-
type
The join type. The different join types are listed here,
ordered from the best type to the worst:
-
system
The table has only one row (= system table). This is a
special case of the const
join type.
-
const
The table has at most one matching row, which is read at
the start of the query. Because there is only one row,
values from the column in this row can be regarded as
constants by the rest of the optimizer.
const
tables are very fast because
they are read only once.
const
is used when you compare all
parts of a PRIMARY KEY
or
UNIQUE
index to constant values. In
the following queries,
tbl_name
can be used as a
const
table:
SELECT * FROM tbl_name
WHERE primary_key
=1;
SELECT * FROM tbl_name
WHERE primary_key_part1
=1 AND primary_key_part2
=2;
-
eq_ref
One row is read from this table for each combination of
rows from the previous tables. Other than the
system
and const
types, this is the best possible join type. It is used
when all parts of an index are used by the join and the
index is a PRIMARY KEY
or
UNIQUE
index.
eq_ref
can be used for indexed
columns that are compared using the =
operator. The comparison value can be a constant or an
expression that uses columns from tables that are read
before this table. In the following examples, MySQL can
use an eq_ref
join to process
ref_table
:
SELECT * FROM ref_table
,other_table
WHERE ref_table
.key_column
=other_table
.column
;
SELECT * FROM ref_table
,other_table
WHERE ref_table
.key_column_part1
=other_table
.column
AND ref_table
.key_column_part2
=1;
-
ref
All rows with matching index values are read from this
table for each combination of rows from the previous
tables. ref
is used if the join uses
only a leftmost prefix of the key or if the key is not a
PRIMARY KEY
or
UNIQUE
index (in other words, if the
join cannot select a single row based on the key value).
If the key that is used matches only a few rows, this is
a good join type.
ref
can be used for indexed columns
that are compared using the =
or
<=>
operator. In the following
examples, MySQL can use a ref
join to
process ref_table
:
SELECT * FROM ref_table
WHERE key_column
=expr
;
SELECT * FROM ref_table
,other_table
WHERE ref_table
.key_column
=other_table
.column
;
SELECT * FROM ref_table
,other_table
WHERE ref_table
.key_column_part1
=other_table
.column
AND ref_table
.key_column_part2
=1;
-
ref_or_null
This join type is like ref
, but with
the addition that MySQL does an extra search for rows
that contain NULL
values. This join
type optimization is used most often in resolving
subqueries. In the following examples, MySQL can use a
ref_or_null
join to process
ref_table
:
SELECT * FROM ref_table
WHERE key_column
=expr
OR key_column
IS NULL;
See Section 7.2.7, “IS NULL
Optimization”.
-
index_merge
This join type indicates that the Index Merge
optimization is used. In this case, the
key
column in the output row contains
a list of indexes used, and key_len
contains a list of the longest key parts for the indexes
used. For more information, see
Section 7.2.6, “Index Merge Optimization”.
-
unique_subquery
This type replaces ref
for some
IN
subqueries of the following form:
value
IN (SELECT primary_key
FROM single_table
WHERE some_expr
)
unique_subquery
is just an index
lookup function that replaces the subquery completely
for better efficiency.
-
index_subquery
This join type is similar to
unique_subquery
. It replaces
IN
subqueries, but it works for
non-unique indexes in subqueries of the following form:
value
IN (SELECT key_column
FROM single_table
WHERE some_expr
)
-
range
Only rows that are in a given range are retrieved, using
an index to select the rows. The key
column in the output row indicates which index is used.
The key_len
contains the longest key
part that was used. The ref
column is
NULL
for this type.
range
can be used when a key column
is compared to a constant using any of the
=
, <>
,
>
, >=
,
<
, <=
,
IS NULL
,
<=>
,
BETWEEN
, or IN
operators:
SELECT * FROM tbl_name
WHERE key_column
= 10;
SELECT * FROM tbl_name
WHERE key_column
BETWEEN 10 and 20;
SELECT * FROM tbl_name
WHERE key_column
IN (10,20,30);
SELECT * FROM tbl_name
WHERE key_part1
= 10 AND key_part2
IN (10,20,30);
-
index
This join type is the same as ALL
,
except that only the index tree is scanned. This usually
is faster than ALL
because the index
file usually is smaller than the data file.
MySQL can use this join type when the query uses only
columns that are part of a single index.
-
ALL
A full table scan is done for each combination of rows
from the previous tables. This is normally not good if
the table is the first table not marked
const
, and usually
very bad in all other cases.
Normally, you can avoid ALL
by adding
indexes that allow row retrieval from the table based on
constant values or column values from earlier tables.
-
possible_keys
The possible_keys
column indicates which
indexes MySQL can choose from use to find the rows in this
table. Note that this column is totally independent of the
order of the tables as displayed in the output from
EXPLAIN
. That means that some of the keys
in possible_keys
might not be usable in
practice with the generated table order.
If this column is NULL
, there are no
relevant indexes. In this case, you may be able to improve
the performance of your query by examining the
WHERE
clause to check whether it refers
to some column or columns that would be suitable for
indexing. If so, create an appropriate index and check the
query with EXPLAIN
again. See
Section 13.1.2, “ALTER TABLE
Syntax”.
To see what indexes a table has, use SHOW INDEX
FROM tbl_name
.
-
key
The key
column indicates the key (index)
that MySQL actually decided to use. The key is
NULL
if no index was chosen. To force
MySQL to use or ignore an index listed in the
possible_keys
column, use FORCE
INDEX
, USE INDEX
, or
IGNORE INDEX
in your query. See
Section 13.2.7, “SELECT
Syntax”.
For MyISAM
and BDB
tables, running ANALYZE TABLE
helps the
optimizer choose better indexes. For
MyISAM
tables, myisamchk
--analyze does the same. See
Section 13.5.2.1, “ANALYZE TABLE
Syntax”, and
Section 5.9.4, “Table Maintenance and Crash Recovery”.
-
key_len
The key_len
column indicates the length
of the key that MySQL decided to use. The length is
NULL
if the key
column
says NULL
. Note that the value of
key_len
enables you to determine how many
parts of a multiple-part key MySQL actually uses.
-
ref
The ref
column shows which columns or
constants are compared to the index named in the
key
column to select rows from the table.
-
rows
The rows
column indicates the number of
rows MySQL believes it must examine to execute the query.
-
Extra
This column contains additional information about how MySQL
resolves the query. Here is an explanation of the values
that can appear in this column:
-
Distinct
MySQL is looking for distinct values, so it stops
searching for more rows for the current row combination
after it has found the first matching row.
-
Not exists
MySQL was able to do a LEFT JOIN
optimization on the query and does not examine more rows
in this table for the previous row combination after it
finds one row that matches the LEFT
JOIN
criteria. Here is an example of the type
of query that can be optimized this way:
SELECT * FROM t1 LEFT JOIN t2 ON t1.id=t2.id
WHERE t2.id IS NULL;
Assume that t2.id
is defined as
NOT NULL
. In this case, MySQL scans
t1
and looks up the rows in
t2
using the values of
t1.id
. If MySQL finds a matching row
in t2
, it knows that
t2.id
can never be
NULL
, and does not scan through the
rest of the rows in t2
that have the
same id
value. In other words, for
each row in t1
, MySQL needs to do
only a single lookup in t2
,
regardless of how many rows actually match in
t2
.
-
range checked for each record (index map:
N
)
MySQL found no good index to use, but found that some of
indexes might be used after column values from preceding
tables are known. For each row combination in the
preceding tables, MySQL checks whether it is possible to
use a range
or
index_merge
access method to retrieve
rows. This is not very fast, but is faster than
performing a join with no index at all. The
applicability criteria are as described in
Section 7.2.5, “Range Optimization”, and
Section 7.2.6, “Index Merge Optimization”, with the
exception that all column values for the preceding table
are known and considered to be constants.
-
Using filesort
MySQL must do an extra pass to find out how to retrieve
the rows in sorted order. The sort is done by going
through all rows according to the join type and storing
the sort key and pointer to the row for all rows that
match the WHERE
clause. The keys then
are sorted and the rows are retrieved in sorted order.
See Section 7.2.12, “ORDER BY
Optimization”.
-
Using index
The column information is retrieved from the table using
only information in the index tree without having to do
an additional seek to read the actual row. This strategy
can be used when the query uses only columns that are
part of a single index.
-
Using temporary
To resolve the query, MySQL needs to create a temporary
table to hold the result. This typically happens if the
query contains GROUP BY
and
ORDER BY
clauses that list columns
differently.
-
Using where
A WHERE
clause is used to restrict
which rows to match against the next table or send to
the client. Unless you specifically intend to fetch or
examine all rows from the table, you may have something
wrong in your query if the Extra
value is not Using where
and the
table join type is ALL
or
index
.
If you want to make your queries as fast as possible,
you should look out for Extra
values
of Using filesort
and Using
temporary
.
-
Using sort_union(...)
, Using
union(...)
, Using
intersect(...)
These indicate how index scans are merged for the
index_merge
join type. See
Section 7.2.6, “Index Merge Optimization”, for more
information.
-
Using index for group-by
Similar to the Using index
way of
accessing a table, Using index for
group-by
indicates that MySQL found an index
that can be used to retrieve all columns of a
GROUP BY
or
DISTINCT
query without any extra disk
access to the actual table. Additionally, the index is
used in the most efficient way so that for each group,
only a few index entries are read. For details, see
Section 7.2.13, “GROUP BY
Optimization”.
-
Using where with pushed condition
This item applies to NDB Cluster
tables only. It means that MySQL
Cluster is using condition
pushdown to improve the efficiency of a
direct comparison (=
) between a
non-indexed column and a constant. In such cases, the
condition is “pushed down” to the cluster's
data nodes where it is evaluated in all partitions
simultaneously. This eliminates the need to send
non-matching rows over the network, and can speed up
such queries by a factor of 5 to 10 times over cases
where condition pushdown could be but is not used.
Suppose that you have a Cluster table defined as
follows:
CREATE TABLE t1 (
a INT,
b INT,
KEY(a)
) ENGINE=NDBCLUSTER;
In this case, condition pushdown can be used with a
query such as this one:
SELECT a,b FROM t1 WHERE b = 10;
This can be seen in the output of EXPLAIN
SELECT
, as shown here:
mysql> EXPLAIN SELECT a,b FROM t1 WHERE b = 10\G
*************************** 1. row ***************************
id: 1
select_type: SIMPLE
table: t1
type: ALL
possible_keys: NULL
key: NULL
key_len: NULL
ref: NULL
rows: 10
Extra: Using where with pushed condition
Condition pushdown cannot be used
with either of these two queries:
SELECT a,b FROM t1 WHERE a = 10;
SELECT a,b FROM t1 WHERE b + 1 = 10;
With regard to the first of these two queries, condition
pushdown is not applicable because an index exists on
column a
. In the case of the second
query, a condition pushdown cannot be employed because
the comparison involving the non-indexed column
b
is an indirect one. (However, it
would apply if you were to reduce b + 1 =
10
to b = 9
in the
WHERE
clause.)
However, a condition pushdown may also be employed when
an indexed column column is compared with a constant
using a >
or
<
operator:
mysql> EXPLAIN SELECT a,b FROM t1 WHERE a<2\G
*************************** 1. row ***************************
id: 1
select_type: SIMPLE
table: t1
type: range
possible_keys: a
key: a
key_len: 5
ref: NULL
rows: 2
Extra: Using where with pushed condition
With regard to condition pushdown, keep in mind that:
Condition pushdown is relevant to MySQL Cluster
only, and does not occur when
executing queries against tables using any other
storage engine.
-
Condition pushdown capability is not used by
default. To enable it, you can start
mysqld with the
--engine-condition-pushdown
option,
or execute the following statement:
SET engine_condition_pushdown=On;
You can get a good indication of how good a join is by taking
the product of the values in the rows
column
of the EXPLAIN
output. This should tell you
roughly how many rows MySQL must examine to execute the query.
If you restrict queries with the
max_join_size
system variable, this row
product also is used to determine which multiple-table
SELECT
statements to execute and which to
abort. See Section 7.5.2, “Tuning Server Parameters”.
The following example shows how a multiple-table join can be
optimized progressively based on the information provided by
EXPLAIN
.
Suppose that you have the SELECT
statement
shown here and that you plan to examine it using
EXPLAIN
:
EXPLAIN SELECT tt.TicketNumber, tt.TimeIn,
tt.ProjectReference, tt.EstimatedShipDate,
tt.ActualShipDate, tt.ClientID,
tt.ServiceCodes, tt.RepetitiveID,
tt.CurrentProcess, tt.CurrentDPPerson,
tt.RecordVolume, tt.DPPrinted, et.COUNTRY,
et_1.COUNTRY, do.CUSTNAME
FROM tt, et, et AS et_1, do
WHERE tt.SubmitTime IS NULL
AND tt.ActualPC = et.EMPLOYID
AND tt.AssignedPC = et_1.EMPLOYID
AND tt.ClientID = do.CUSTNMBR;
For this example, make the following assumptions:
-
The columns being compared have been declared as follows:
-
The tables have the following indexes:
The tt.ActualPC
values are not evenly
distributed.
Initially, before any optimizations have been performed, the
EXPLAIN
statement produces the following
information:
table type possible_keys key key_len ref rows Extra
et ALL PRIMARY NULL NULL NULL 74
do ALL PRIMARY NULL NULL NULL 2135
et_1 ALL PRIMARY NULL NULL NULL 74
tt ALL AssignedPC, NULL NULL NULL 3872
ClientID,
ActualPC
range checked for each record (key map: 35)
Because type
is ALL
for
each table, this output indicates that MySQL is generating a
Cartesian product of all the tables; that is, every combination
of rows. This takes quite a long time, because the product of
the number of rows in each table must be examined. For the case
at hand, this product is 74 × 2135 × 74 × 3872
= 45,268,558,720 rows. If the tables were bigger, you can only
imagine how long it would take.
One problem here is that MySQL can use indexes on columns more
efficiently if they are declared as the same type and size. In
this context, VARCHAR
and
CHAR
are considered the same if they are
declared as the same size. tt.ActualPC
is
declared as CHAR(10)
and
et.EMPLOYID
is CHAR(15)
,
so there is a length mismatch.
To fix this disparity between column lengths, use ALTER
TABLE
to lengthen ActualPC
from 10
characters to 15 characters:
mysql> ALTER TABLE tt MODIFY ActualPC VARCHAR(15);
Now tt.ActualPC
and
et.EMPLOYID
are both
VARCHAR(15)
. Executing the
EXPLAIN
statement again produces this result:
table type possible_keys key key_len ref rows Extra
tt ALL AssignedPC, NULL NULL NULL 3872 Using
ClientID, where
ActualPC
do ALL PRIMARY NULL NULL NULL 2135
range checked for each record (key map: 1)
et_1 ALL PRIMARY NULL NULL NULL 74
range checked for each record (key map: 1)
et eq_ref PRIMARY PRIMARY 15 tt.ActualPC 1
This is not perfect, but is much better: The product of the
rows
values is less by a factor of 74. This
version executes in a couple of seconds.
A second alteration can be made to eliminate the column length
mismatches for the tt.AssignedPC =
et_1.EMPLOYID
and tt.ClientID =
do.CUSTNMBR
comparisons:
mysql> ALTER TABLE tt MODIFY AssignedPC VARCHAR(15),
-> MODIFY ClientID VARCHAR(15);
After that modification, EXPLAIN
produces the
output shown here:
table type possible_keys key key_len ref rows Extra
et ALL PRIMARY NULL NULL NULL 74
tt ref AssignedPC, ActualPC 15 et.EMPLOYID 52 Using
ClientID, where
ActualPC
et_1 eq_ref PRIMARY PRIMARY 15 tt.AssignedPC 1
do eq_ref PRIMARY PRIMARY 15 tt.ClientID 1
At this point, the query is optimized almost as well as
possible. The remaining problem is that, by default, MySQL
assumes that values in the tt.ActualPC
column
are evenly distributed, and that is not the case for the
tt
table. Fortunately, it is easy to tell
MySQL to analyze the key distribution:
mysql> ANALYZE TABLE tt;
With the additional index information, the join is perfect and
EXPLAIN
produces this result:
table type possible_keys key key_len ref rows Extra
tt ALL AssignedPC NULL NULL NULL 3872 Using
ClientID, where
ActualPC
et eq_ref PRIMARY PRIMARY 15 tt.ActualPC 1
et_1 eq_ref PRIMARY PRIMARY 15 tt.AssignedPC 1
do eq_ref PRIMARY PRIMARY 15 tt.ClientID 1
Note that the rows
column in the output from
EXPLAIN
is an educated guess from the MySQL
join optimizer. You should check whether the numbers are even
close to the truth by comparing the rows
product with the actual number of rows that the query returns.
If the numbers are quite different, you might get better
performance by using STRAIGHT_JOIN
in your
SELECT
statement and trying to list the
tables in a different order in the FROM
clause.