mirror of
https://github.com/a52xq-dev/kernel.git
synced 2026-10-18 21:58:11 +07:00
Some big changes this month, headlined by the addition of a new formatted
documentation mechanism based on the Sphinx system. The objectives here are to make it easier to create better-integrated (and more attractive) documents while (eventually) dumping our one-of-a-kind, cobbled-together system for something that is widely used and maintained by others. There's a fair amount of information what's being done, why, and how to use it in: https://lwn.net/Articles/692704/ https://lwn.net/Articles/692705/ Closer to home, Documentation/kernel-documentation.rst describes how it works. For now, the new system exists alongside the old one; you should soon see the GPU documentation converted over in the DRM pull and some significant media conversion work as well. Once all the docs have been moved over and we're convinced that the rough edges (of which are are a few) have been smoothed over, the DocBook-based stuff should go away. Primary credit is to Jani Nikula for doing the heavy lifting to make this stuff actually work; there has also been notable effort from Markus Heiser, Daniel Vetter, and Mauro Carvalho Chehab. Expect a couple of conflicts on the new index.rst file over the course of the merge window; they are trivially resolvable. That file may be a bit of a conflict magnet in the short term, but I don't expect that situation to last for any real length of time. Beyond that, of course, we have the usual collection of tweaks, updates, and typo fixes. -----BEGIN PGP SIGNATURE----- Version: GnuPG v1 iQIcBAABAgAGBQJXlT8bAAoJEI3ONVYwIuV6IF0QAM7qpEgAveXgwzGr18ReftjB ryyPXdLNX8R7fQkUAu85Znlw93k4+7QxHseBgZ1I3t9IhjvzemidOKfj96SJ0DVE zco5FUTO1TPfO2uG8asNzlXSrwktrEZoFTraRIYpCjAAimg/UEQAM4xgNOJfmW39 8eLg6dhnzqnfCH6M+Q7AZPLmqNSOfA2qNFqHNs7VCbwjLQC0l4yw4PHOsdX0QhCL NtlukHOhB4CwaELDl6lZOQgnz79e4pDljFd/wA/vVpdq7ev8NnOgVVt93M2fUvLX Bdfzo2OtZg7LJnWtpQl2LJq2MJJdOmgIw+ls6cB5gvxV3+CCIndE/6IIbimC/lZi DiaaqVU/EV3AemBDbqErO4k3ZvMT2ZTaSx1OQoLpIwNAsAkEOkb+t2hcwrTqhxg1 kEIXEEboXwStBtH3iar2YiGHQWUhia4mxfv5n4Dvamaca4tGWXHWxSRjbRcaJ1ef v1kt6ymjwr2sd9116EUxngfYONwlrhk5F87D/dC2eQzEyS0z7+F8wWb/h2gsAaU1 6oVCoQjTe2KAc/7dHINMSIjg2TfVfHS5XIhWu5z6wyxRb8ey9QuKLQYlOTV/L3Ms aR4O6qvDIsIbOvEd+hHSIbeG7JOPd0LdSI4tdo4d9kquW2OJ6TmmOr/ihbJt430v s/Qxqwgme/8DJd3KcV6j =4fDK -----END PGP SIGNATURE----- Merge tag 'docs-for-linus' of git://git.lwn.net/linux Pull documentation updates from Jonathan Corbet: "Some big changes this month, headlined by the addition of a new formatted documentation mechanism based on the Sphinx system. The objectives here are to make it easier to create better-integrated (and more attractive) documents while (eventually) dumping our one-of-a-kind, cobbled-together system for something that is widely used and maintained by others. There's a fair amount of information what's being done, why, and how to use it in: https://lwn.net/Articles/692704/ https://lwn.net/Articles/692705/ Closer to home, Documentation/kernel-documentation.rst describes how it works. For now, the new system exists alongside the old one; you should soon see the GPU documentation converted over in the DRM pull and some significant media conversion work as well. Once all the docs have been moved over and we're convinced that the rough edges (of which are are a few) have been smoothed over, the DocBook-based stuff should go away. Primary credit is to Jani Nikula for doing the heavy lifting to make this stuff actually work; there has also been notable effort from Markus Heiser, Daniel Vetter, and Mauro Carvalho Chehab. Expect a couple of conflicts on the new index.rst file over the course of the merge window; they are trivially resolvable. That file may be a bit of a conflict magnet in the short term, but I don't expect that situation to last for any real length of time. Beyond that, of course, we have the usual collection of tweaks, updates, and typo fixes" * tag 'docs-for-linus' of git://git.lwn.net/linux: (77 commits) doc-rst: kernel-doc: fix handling of address_space tags Revert "doc/sphinx: Enable keep_warnings" doc-rst: kernel-doc directive, fix state machine reporter docs: deprecate kernel-doc-nano-HOWTO.txt doc/sphinx: Enable keep_warnings Documentation: add watermark_scale_factor to the list of vm systcl file kernel-doc: Fix up warning output docs: Get rid of some kernel-documentation warnings doc-rst: add an option to ignore DocBooks when generating docs workqueue: Fix a typo in workqueue.txt Doc: ocfs: Fix typo in filesystems/ocfs2-online-filecheck.txt Documentation/sphinx: skip build if user requested specific DOCBOOKS Documentation: add cleanmediadocs to the documentation targets Add .pyc files to .gitignore Doc: PM: Fix a typo in intel_powerclamp.txt doc-rst: flat-table directive - initial implementation Documentation: add meta-documentation for Sphinx and kernel-doc Documentation: tiny typo fix in usb/gadget_multi.txt Documentation: fix wrong value in md.txt bcache: documentation formatting, edited for clarity, stripe alignment notes ...
This commit is contained in:
Linus Torvalds
commit
0f776dc377
30 changed files with 2653 additions and 430 deletions
2
Documentation/.gitignore
vendored
Normal file
2
Documentation/.gitignore
vendored
Normal file
|
|
@ -0,0 +1,2 @@
|
|||
output
|
||||
*.pyc
|
||||
|
|
@ -255,10 +255,10 @@ kbuild/
|
|||
- directory with info about the kernel build process.
|
||||
kdump/
|
||||
- directory with mini HowTo on getting the crash dump code to work.
|
||||
kernel-doc-nano-HOWTO.txt
|
||||
- mini HowTo on generation and location of kernel documentation files.
|
||||
kernel-docs.txt
|
||||
- listing of various WWW + books that document kernel internals.
|
||||
kernel-documentation.rst
|
||||
- how to write and format reStructuredText kernel documentation
|
||||
kernel-parameters.txt
|
||||
- summary listing of command line / boot prompt args for the kernel.
|
||||
kernel-per-CPU-kthreads.txt
|
||||
|
|
|
|||
|
|
@ -458,7 +458,7 @@ of the function, telling people what it does, and possibly WHY it does
|
|||
it.
|
||||
|
||||
When commenting the kernel API functions, please use the kernel-doc format.
|
||||
See the files Documentation/kernel-doc-nano-HOWTO.txt and scripts/kernel-doc
|
||||
See the files Documentation/kernel-documentation.rst and scripts/kernel-doc
|
||||
for details.
|
||||
|
||||
Linux style for comments is the C89 "/* ... */" style.
|
||||
|
|
|
|||
|
|
@ -6,6 +6,8 @@
|
|||
# To add a new book the only step required is to add the book to the
|
||||
# list of DOCBOOKS.
|
||||
|
||||
ifeq ($(IGNORE_DOCBOOKS),)
|
||||
|
||||
DOCBOOKS := z8530book.xml device-drivers.xml \
|
||||
kernel-hacking.xml kernel-locking.xml deviceiobook.xml \
|
||||
writing_usb_driver.xml networking.xml \
|
||||
|
|
@ -33,10 +35,6 @@ PDF_METHOD = $(prefer-db2x)
|
|||
PS_METHOD = $(prefer-db2x)
|
||||
|
||||
|
||||
###
|
||||
# The targets that may be used.
|
||||
PHONY += xmldocs sgmldocs psdocs pdfdocs htmldocs mandocs installmandocs cleandocs
|
||||
|
||||
targets += $(DOCBOOKS)
|
||||
BOOKS := $(addprefix $(obj)/,$(DOCBOOKS))
|
||||
xmldocs: $(BOOKS)
|
||||
|
|
@ -63,6 +61,9 @@ installmandocs: mandocs
|
|||
sort -k 2 -k 1 | uniq -f 1 | sed -e 's: :/:' | \
|
||||
xargs install -m 644 -t /usr/local/man/man9/
|
||||
|
||||
# no-op for the DocBook toolchain
|
||||
epubdocs:
|
||||
|
||||
###
|
||||
#External programs used
|
||||
KERNELDOCXMLREF = $(srctree)/scripts/kernel-doc-xml-ref
|
||||
|
|
@ -216,10 +217,24 @@ silent_gen_xml = :
|
|||
-e "s/>/\\>/g"; \
|
||||
echo "</programlisting>") > $@
|
||||
|
||||
else
|
||||
|
||||
# Needed, due to cleanmediadocs
|
||||
include Documentation/DocBook/media/Makefile
|
||||
|
||||
htmldocs:
|
||||
pdfdocs:
|
||||
psdocs:
|
||||
xmldocs:
|
||||
installmandocs:
|
||||
|
||||
endif # IGNORE_DOCBOOKS
|
||||
|
||||
|
||||
###
|
||||
# Help targets as used by the top-level makefile
|
||||
dochelp:
|
||||
@echo ' Linux kernel internal documentation in different formats:'
|
||||
@echo ' Linux kernel internal documentation in different formats (DocBook):'
|
||||
@echo ' htmldocs - HTML'
|
||||
@echo ' pdfdocs - PDF'
|
||||
@echo ' psdocs - Postscript'
|
||||
|
|
@ -228,8 +243,11 @@ dochelp:
|
|||
@echo ' installmandocs - install man pages generated by mandocs'
|
||||
@echo ' cleandocs - clean all generated DocBook files'
|
||||
@echo
|
||||
@echo 'make DOCBOOKS="s1.xml s2.xml" [target] Generate only docs s1.xml s2.xml'
|
||||
@echo ' make DOCBOOKS="s1.xml s2.xml" [target] Generate only docs s1.xml s2.xml'
|
||||
@echo ' valid values for DOCBOOKS are: $(DOCBOOKS)'
|
||||
@echo
|
||||
@echo " make IGNORE_DOCBOOKS=1 [target] Don't generate docs from Docbook"
|
||||
@echo ' This is useful to generate only the ReST docs (Sphinx)'
|
||||
|
||||
|
||||
###
|
||||
|
|
|
|||
78
Documentation/Makefile.sphinx
Normal file
78
Documentation/Makefile.sphinx
Normal file
|
|
@ -0,0 +1,78 @@
|
|||
# -*- makefile -*-
|
||||
# Makefile for Sphinx documentation
|
||||
#
|
||||
|
||||
# You can set these variables from the command line.
|
||||
SPHINXBUILD = sphinx-build
|
||||
SPHINXOPTS =
|
||||
PAPER =
|
||||
BUILDDIR = $(obj)/output
|
||||
|
||||
# User-friendly check for sphinx-build
|
||||
HAVE_SPHINX := $(shell if which $(SPHINXBUILD) >/dev/null 2>&1; then echo 1; else echo 0; fi)
|
||||
|
||||
ifeq ($(HAVE_SPHINX),0)
|
||||
|
||||
.DEFAULT:
|
||||
$(warning The '$(SPHINXBUILD)' command was not found. Make sure you have Sphinx installed and in PATH, or set the SPHINXBUILD make variable to point to the full path of the '$(SPHINXBUILD)' executable.)
|
||||
@echo " SKIP Sphinx $@ target."
|
||||
|
||||
else ifneq ($(DOCBOOKS),)
|
||||
|
||||
# Skip Sphinx build if the user explicitly requested DOCBOOKS.
|
||||
.DEFAULT:
|
||||
@echo " SKIP Sphinx $@ target (DOCBOOKS specified)."
|
||||
|
||||
else # HAVE_SPHINX
|
||||
|
||||
# User-friendly check for rst2pdf
|
||||
HAVE_RST2PDF := $(shell if python -c "import rst2pdf" >/dev/null 2>&1; then echo 1; else echo 0; fi)
|
||||
|
||||
# Internal variables.
|
||||
PAPEROPT_a4 = -D latex_paper_size=a4
|
||||
PAPEROPT_letter = -D latex_paper_size=letter
|
||||
KERNELDOC = $(srctree)/scripts/kernel-doc
|
||||
KERNELDOC_CONF = -D kerneldoc_srctree=$(srctree) -D kerneldoc_bin=$(KERNELDOC)
|
||||
ALLSPHINXOPTS = -D version=$(KERNELVERSION) -D release=$(KERNELRELEASE) -d $(BUILDDIR)/.doctrees $(KERNELDOC_CONF) $(PAPEROPT_$(PAPER)) -c $(srctree)/$(src) $(SPHINXOPTS) $(srctree)/$(src)
|
||||
# the i18n builder cannot share the environment and doctrees with the others
|
||||
I18NSPHINXOPTS = $(PAPEROPT_$(PAPER)) $(SPHINXOPTS) .
|
||||
|
||||
quiet_cmd_sphinx = SPHINX $@
|
||||
cmd_sphinx = $(SPHINXBUILD) -b $2 $(ALLSPHINXOPTS) $(BUILDDIR)/$2
|
||||
|
||||
htmldocs:
|
||||
$(call cmd,sphinx,html)
|
||||
|
||||
pdfdocs:
|
||||
ifeq ($(HAVE_RST2PDF),0)
|
||||
$(warning The Python 'rst2pdf' module was not found. Make sure you have the module installed to produce PDF output.)
|
||||
@echo " SKIP Sphinx $@ target."
|
||||
else # HAVE_RST2PDF
|
||||
$(call cmd,sphinx,pdf)
|
||||
endif # HAVE_RST2PDF
|
||||
|
||||
epubdocs:
|
||||
$(call cmd,sphinx,epub)
|
||||
|
||||
xmldocs:
|
||||
$(call cmd,sphinx,xml)
|
||||
|
||||
# no-ops for the Sphinx toolchain
|
||||
sgmldocs:
|
||||
psdocs:
|
||||
mandocs:
|
||||
installmandocs:
|
||||
cleanmediadocs:
|
||||
|
||||
cleandocs:
|
||||
$(Q)rm -rf $(BUILDDIR)
|
||||
|
||||
dochelp:
|
||||
@echo ' Linux kernel internal documentation in different formats (Sphinx):'
|
||||
@echo ' htmldocs - HTML'
|
||||
@echo ' pdfdocs - PDF'
|
||||
@echo ' epubdocs - EPUB'
|
||||
@echo ' xmldocs - XML'
|
||||
@echo ' cleandocs - clean all generated files'
|
||||
|
||||
endif # HAVE_SPHINX
|
||||
|
|
@ -1,4 +1,4 @@
|
|||
Say you've got a big slow raid 6, and an X-25E or three. Wouldn't it be
|
||||
Say you've got a big slow raid 6, and an ssd or three. Wouldn't it be
|
||||
nice if you could use them as cache... Hence bcache.
|
||||
|
||||
Wiki and git repositories are at:
|
||||
|
|
@ -8,7 +8,7 @@ Wiki and git repositories are at:
|
|||
|
||||
It's designed around the performance characteristics of SSDs - it only allocates
|
||||
in erase block sized buckets, and it uses a hybrid btree/log to track cached
|
||||
extants (which can be anywhere from a single sector to the bucket size). It's
|
||||
extents (which can be anywhere from a single sector to the bucket size). It's
|
||||
designed to avoid random writes at all costs; it fills up an erase block
|
||||
sequentially, then issues a discard before reusing it.
|
||||
|
||||
|
|
@ -55,7 +55,10 @@ immediately. Without udev, you can manually register devices like this:
|
|||
Registering the backing device makes the bcache device show up in /dev; you can
|
||||
now format it and use it as normal. But the first time using a new bcache
|
||||
device, it'll be running in passthrough mode until you attach it to a cache.
|
||||
See the section on attaching.
|
||||
If you are thinking about using bcache later, it is recommended to setup all your
|
||||
slow devices as bcache backing devices without a cache, and you can choose to add
|
||||
a caching device later.
|
||||
See 'ATTACHING' section below.
|
||||
|
||||
The devices show up as:
|
||||
|
||||
|
|
@ -72,12 +75,14 @@ To get started:
|
|||
mount /dev/bcache0 /mnt
|
||||
|
||||
You can control bcache devices through sysfs at /sys/block/bcache<N>/bcache .
|
||||
You can also control them through /sys/fs//bcache/<cset-uuid>/ .
|
||||
|
||||
Cache devices are managed as sets; multiple caches per set isn't supported yet
|
||||
but will allow for mirroring of metadata and dirty data in the future. Your new
|
||||
cache set shows up as /sys/fs/bcache/<UUID>
|
||||
|
||||
ATTACHING:
|
||||
ATTACHING
|
||||
---------
|
||||
|
||||
After your cache device and backing device are registered, the backing device
|
||||
must be attached to your cache set to enable caching. Attaching a backing
|
||||
|
|
@ -105,7 +110,8 @@ but all the cached data will be invalidated. If there was dirty data in the
|
|||
cache, don't expect the filesystem to be recoverable - you will have massive
|
||||
filesystem corruption, though ext4's fsck does work miracles.
|
||||
|
||||
ERROR HANDLING:
|
||||
ERROR HANDLING
|
||||
--------------
|
||||
|
||||
Bcache tries to transparently handle IO errors to/from the cache device without
|
||||
affecting normal operation; if it sees too many errors (the threshold is
|
||||
|
|
@ -127,12 +133,181 @@ the backing devices to passthrough mode.
|
|||
writeback mode). It currently doesn't do anything intelligent if it fails to
|
||||
read some of the dirty data, though.
|
||||
|
||||
TROUBLESHOOTING PERFORMANCE:
|
||||
|
||||
HOWTO/COOKBOOK
|
||||
--------------
|
||||
|
||||
A) Starting a bcache with a missing caching device
|
||||
|
||||
If registering the backing device doesn't help, it's already there, you just need
|
||||
to force it to run without the cache:
|
||||
host:~# echo /dev/sdb1 > /sys/fs/bcache/register
|
||||
[ 119.844831] bcache: register_bcache() error opening /dev/sdb1: device already registered
|
||||
|
||||
Next, you try to register your caching device if it's present. However
|
||||
if it's absent, or registration fails for some reason, you can still
|
||||
start your bcache without its cache, like so:
|
||||
host:/sys/block/sdb/sdb1/bcache# echo 1 > running
|
||||
|
||||
Note that this may cause data loss if you were running in writeback mode.
|
||||
|
||||
|
||||
B) Bcache does not find its cache
|
||||
|
||||
host:/sys/block/md5/bcache# echo 0226553a-37cf-41d5-b3ce-8b1e944543a8 > attach
|
||||
[ 1933.455082] bcache: bch_cached_dev_attach() Couldn't find uuid for md5 in set
|
||||
[ 1933.478179] bcache: __cached_dev_store() Can't attach 0226553a-37cf-41d5-b3ce-8b1e944543a8
|
||||
[ 1933.478179] : cache set not found
|
||||
|
||||
In this case, the caching device was simply not registered at boot
|
||||
or disappeared and came back, and needs to be (re-)registered:
|
||||
host:/sys/block/md5/bcache# echo /dev/sdh2 > /sys/fs/bcache/register
|
||||
|
||||
|
||||
C) Corrupt bcache crashes the kernel at device registration time:
|
||||
|
||||
This should never happen. If it does happen, then you have found a bug!
|
||||
Please report it to the bcache development list: linux-bcache@vger.kernel.org
|
||||
|
||||
Be sure to provide as much information that you can including kernel dmesg
|
||||
output if available so that we may assist.
|
||||
|
||||
|
||||
D) Recovering data without bcache:
|
||||
|
||||
If bcache is not available in the kernel, a filesystem on the backing
|
||||
device is still available at an 8KiB offset. So either via a loopdev
|
||||
of the backing device created with --offset 8K, or any value defined by
|
||||
--data-offset when you originally formatted bcache with `make-bcache`.
|
||||
|
||||
For example:
|
||||
losetup -o 8192 /dev/loop0 /dev/your_bcache_backing_dev
|
||||
|
||||
This should present your unmodified backing device data in /dev/loop0
|
||||
|
||||
If your cache is in writethrough mode, then you can safely discard the
|
||||
cache device without loosing data.
|
||||
|
||||
|
||||
E) Wiping a cache device
|
||||
|
||||
host:~# wipefs -a /dev/sdh2
|
||||
16 bytes were erased at offset 0x1018 (bcache)
|
||||
they were: c6 85 73 f6 4e 1a 45 ca 82 65 f5 7f 48 ba 6d 81
|
||||
|
||||
After you boot back with bcache enabled, you recreate the cache and attach it:
|
||||
host:~# make-bcache -C /dev/sdh2
|
||||
UUID: 7be7e175-8f4c-4f99-94b2-9c904d227045
|
||||
Set UUID: 5bc072a8-ab17-446d-9744-e247949913c1
|
||||
version: 0
|
||||
nbuckets: 106874
|
||||
block_size: 1
|
||||
bucket_size: 1024
|
||||
nr_in_set: 1
|
||||
nr_this_dev: 0
|
||||
first_bucket: 1
|
||||
[ 650.511912] bcache: run_cache_set() invalidating existing data
|
||||
[ 650.549228] bcache: register_cache() registered cache device sdh2
|
||||
|
||||
start backing device with missing cache:
|
||||
host:/sys/block/md5/bcache# echo 1 > running
|
||||
|
||||
attach new cache:
|
||||
host:/sys/block/md5/bcache# echo 5bc072a8-ab17-446d-9744-e247949913c1 > attach
|
||||
[ 865.276616] bcache: bch_cached_dev_attach() Caching md5 as bcache0 on set 5bc072a8-ab17-446d-9744-e247949913c1
|
||||
|
||||
|
||||
F) Remove or replace a caching device
|
||||
|
||||
host:/sys/block/sda/sda7/bcache# echo 1 > detach
|
||||
[ 695.872542] bcache: cached_dev_detach_finish() Caching disabled for sda7
|
||||
|
||||
host:~# wipefs -a /dev/nvme0n1p4
|
||||
wipefs: error: /dev/nvme0n1p4: probing initialization failed: Device or resource busy
|
||||
Ooops, it's disabled, but not unregistered, so it's still protected
|
||||
|
||||
We need to go and unregister it:
|
||||
host:/sys/fs/bcache/b7ba27a1-2398-4649-8ae3-0959f57ba128# ls -l cache0
|
||||
lrwxrwxrwx 1 root root 0 Feb 25 18:33 cache0 -> ../../../devices/pci0000:00/0000:00:1d.0/0000:70:00.0/nvme/nvme0/nvme0n1/nvme0n1p4/bcache/
|
||||
host:/sys/fs/bcache/b7ba27a1-2398-4649-8ae3-0959f57ba128# echo 1 > stop
|
||||
kernel: [ 917.041908] bcache: cache_set_free() Cache set b7ba27a1-2398-4649-8ae3-0959f57ba128 unregistered
|
||||
|
||||
Now we can wipe it:
|
||||
host:~# wipefs -a /dev/nvme0n1p4
|
||||
/dev/nvme0n1p4: 16 bytes were erased at offset 0x00001018 (bcache): c6 85 73 f6 4e 1a 45 ca 82 65 f5 7f 48 ba 6d 81
|
||||
|
||||
|
||||
G) dm-crypt and bcache
|
||||
|
||||
First setup bcache unencrypted and then install dmcrypt on top of
|
||||
/dev/bcache<N> This will work faster than if you dmcrypt both the backing
|
||||
and caching devices and then install bcache on top. [benchmarks?]
|
||||
|
||||
|
||||
H) Stop/free a registered bcache to wipe and/or recreate it
|
||||
|
||||
Suppose that you need to free up all bcache references so that you can
|
||||
fdisk run and re-register a changed partition table, which won't work
|
||||
if there are any active backing or caching devices left on it:
|
||||
|
||||
1) Is it present in /dev/bcache* ? (there are times where it won't be)
|
||||
|
||||
If so, it's easy:
|
||||
host:/sys/block/bcache0/bcache# echo 1 > stop
|
||||
|
||||
2) But if your backing device is gone, this won't work:
|
||||
host:/sys/block/bcache0# cd bcache
|
||||
bash: cd: bcache: No such file or directory
|
||||
|
||||
In this case, you may have to unregister the dmcrypt block device that
|
||||
references this bcache to free it up:
|
||||
host:~# dmsetup remove oldds1
|
||||
bcache: bcache_device_free() bcache0 stopped
|
||||
bcache: cache_set_free() Cache set 5bc072a8-ab17-446d-9744-e247949913c1 unregistered
|
||||
|
||||
This causes the backing bcache to be removed from /sys/fs/bcache and
|
||||
then it can be reused. This would be true of any block device stacking
|
||||
where bcache is a lower device.
|
||||
|
||||
3) In other cases, you can also look in /sys/fs/bcache/:
|
||||
|
||||
host:/sys/fs/bcache# ls -l */{cache?,bdev?}
|
||||
lrwxrwxrwx 1 root root 0 Mar 5 09:39 0226553a-37cf-41d5-b3ce-8b1e944543a8/bdev1 -> ../../../devices/virtual/block/dm-1/bcache/
|
||||
lrwxrwxrwx 1 root root 0 Mar 5 09:39 0226553a-37cf-41d5-b3ce-8b1e944543a8/cache0 -> ../../../devices/virtual/block/dm-4/bcache/
|
||||
lrwxrwxrwx 1 root root 0 Mar 5 09:39 5bc072a8-ab17-446d-9744-e247949913c1/cache0 -> ../../../devices/pci0000:00/0000:00:01.0/0000:01:00.0/ata10/host9/target9:0:0/9:0:0:0/block/sdl/sdl2/bcache/
|
||||
|
||||
The device names will show which UUID is relevant, cd in that directory
|
||||
and stop the cache:
|
||||
host:/sys/fs/bcache/5bc072a8-ab17-446d-9744-e247949913c1# echo 1 > stop
|
||||
|
||||
This will free up bcache references and let you reuse the partition for
|
||||
other purposes.
|
||||
|
||||
|
||||
|
||||
TROUBLESHOOTING PERFORMANCE
|
||||
---------------------------
|
||||
|
||||
Bcache has a bunch of config options and tunables. The defaults are intended to
|
||||
be reasonable for typical desktop and server workloads, but they're not what you
|
||||
want for getting the best possible numbers when benchmarking.
|
||||
|
||||
- Backing device alignment
|
||||
|
||||
The default metadata size in bcache is 8k. If your backing device is
|
||||
RAID based, then be sure to align this by a multiple of your stride
|
||||
width using `make-bcache --data-offset`. If you intend to expand your
|
||||
disk array in the future, then multiply a series of primes by your
|
||||
raid stripe size to get the disk multiples that you would like.
|
||||
|
||||
For example: If you have a 64k stripe size, then the following offset
|
||||
would provide alignment for many common RAID5 data spindle counts:
|
||||
64k * 2*2*2*3*3*5*7 bytes = 161280k
|
||||
|
||||
That space is wasted, but for only 157.5MB you can grow your RAID 5
|
||||
volume to the following data-spindle counts without re-aligning:
|
||||
3,4,5,6,7,8,9,10,12,14,15,18,20,21 ...
|
||||
|
||||
- Bad write performance
|
||||
|
||||
If write performance is not what you expected, you probably wanted to be
|
||||
|
|
@ -140,7 +315,7 @@ want for getting the best possible numbers when benchmarking.
|
|||
maturity, but simply because in writeback mode you'll lose data if something
|
||||
happens to your SSD)
|
||||
|
||||
# echo writeback > /sys/block/bcache0/cache_mode
|
||||
# echo writeback > /sys/block/bcache0/bcache/cache_mode
|
||||
|
||||
- Bad performance, or traffic not going to the SSD that you'd expect
|
||||
|
||||
|
|
@ -193,7 +368,9 @@ want for getting the best possible numbers when benchmarking.
|
|||
Solution: warm the cache by doing writes, or use the testing branch (there's
|
||||
a fix for the issue there).
|
||||
|
||||
SYSFS - BACKING DEVICE:
|
||||
|
||||
SYSFS - BACKING DEVICE
|
||||
----------------------
|
||||
|
||||
Available at /sys/block/<bdev>/bcache, /sys/block/bcache*/bcache and
|
||||
(if attached) /sys/fs/bcache/<cset-uuid>/bdev*
|
||||
|
|
@ -238,7 +415,7 @@ sequential_merge
|
|||
against all new requests to determine which new requests are sequential
|
||||
continuations of previous requests for the purpose of determining sequential
|
||||
cutoff. This is necessary if the sequential cutoff value is greater than the
|
||||
maximum acceptable sequential size for any single request.
|
||||
maximum acceptable sequential size for any single request.
|
||||
|
||||
state
|
||||
The backing device can be in one of four different states:
|
||||
|
|
@ -325,7 +502,7 @@ bucket_size
|
|||
Size of buckets
|
||||
|
||||
cache<0..n>
|
||||
Symlink to each of the cache devices comprising this cache set.
|
||||
Symlink to each of the cache devices comprising this cache set.
|
||||
|
||||
cache_available_percent
|
||||
Percentage of cache device which doesn't contain dirty data, and could
|
||||
|
|
|
|||
414
Documentation/conf.py
Normal file
414
Documentation/conf.py
Normal file
|
|
@ -0,0 +1,414 @@
|
|||
# -*- coding: utf-8 -*-
|
||||
#
|
||||
# The Linux Kernel documentation build configuration file, created by
|
||||
# sphinx-quickstart on Fri Feb 12 13:51:46 2016.
|
||||
#
|
||||
# This file is execfile()d with the current directory set to its
|
||||
# containing dir.
|
||||
#
|
||||
# Note that not all possible configuration values are present in this
|
||||
# autogenerated file.
|
||||
#
|
||||
# All configuration values have a default; values that are commented out
|
||||
# serve to show the default.
|
||||
|
||||
import sys
|
||||
import os
|
||||
|
||||
# If extensions (or modules to document with autodoc) are in another directory,
|
||||
# add these directories to sys.path here. If the directory is relative to the
|
||||
# documentation root, use os.path.abspath to make it absolute, like shown here.
|
||||
sys.path.insert(0, os.path.abspath('sphinx'))
|
||||
|
||||
# -- General configuration ------------------------------------------------
|
||||
|
||||
# If your documentation needs a minimal Sphinx version, state it here.
|
||||
#needs_sphinx = '1.0'
|
||||
|
||||
# Add any Sphinx extension module names here, as strings. They can be
|
||||
# extensions coming with Sphinx (named 'sphinx.ext.*') or your custom
|
||||
# ones.
|
||||
extensions = ['kernel-doc', 'rstFlatTable']
|
||||
|
||||
# Gracefully handle missing rst2pdf.
|
||||
try:
|
||||
import rst2pdf
|
||||
extensions += ['rst2pdf.pdfbuilder']
|
||||
except ImportError:
|
||||
pass
|
||||
|
||||
# Add any paths that contain templates here, relative to this directory.
|
||||
templates_path = ['_templates']
|
||||
|
||||
# The suffix(es) of source filenames.
|
||||
# You can specify multiple suffix as a list of string:
|
||||
# source_suffix = ['.rst', '.md']
|
||||
source_suffix = '.rst'
|
||||
|
||||
# The encoding of source files.
|
||||
#source_encoding = 'utf-8-sig'
|
||||
|
||||
# The master toctree document.
|
||||
master_doc = 'index'
|
||||
|
||||
# General information about the project.
|
||||
project = 'The Linux Kernel'
|
||||
copyright = '2016, The kernel development community'
|
||||
author = 'The kernel development community'
|
||||
|
||||
# The version info for the project you're documenting, acts as replacement for
|
||||
# |version| and |release|, also used in various other places throughout the
|
||||
# built documents.
|
||||
#
|
||||
# In a normal build, version and release are are set to KERNELVERSION and
|
||||
# KERNELRELEASE, respectively, from the Makefile via Sphinx command line
|
||||
# arguments.
|
||||
#
|
||||
# The following code tries to extract the information by reading the Makefile,
|
||||
# when Sphinx is run directly (e.g. by Read the Docs).
|
||||
try:
|
||||
makefile_version = None
|
||||
makefile_patchlevel = None
|
||||
for line in open('../Makefile'):
|
||||
key, val = [x.strip() for x in line.split('=', 2)]
|
||||
if key == 'VERSION':
|
||||
makefile_version = val
|
||||
elif key == 'PATCHLEVEL':
|
||||
makefile_patchlevel = val
|
||||
if makefile_version and makefile_patchlevel:
|
||||
break
|
||||
except:
|
||||
pass
|
||||
finally:
|
||||
if makefile_version and makefile_patchlevel:
|
||||
version = release = makefile_version + '.' + makefile_patchlevel
|
||||
else:
|
||||
sys.stderr.write('Warning: Could not extract kernel version\n')
|
||||
version = release = "unknown version"
|
||||
|
||||
# The language for content autogenerated by Sphinx. Refer to documentation
|
||||
# for a list of supported languages.
|
||||
#
|
||||
# This is also used if you do content translation via gettext catalogs.
|
||||
# Usually you set "language" from the command line for these cases.
|
||||
language = None
|
||||
|
||||
# There are two options for replacing |today|: either, you set today to some
|
||||
# non-false value, then it is used:
|
||||
#today = ''
|
||||
# Else, today_fmt is used as the format for a strftime call.
|
||||
#today_fmt = '%B %d, %Y'
|
||||
|
||||
# List of patterns, relative to source directory, that match files and
|
||||
# directories to ignore when looking for source files.
|
||||
exclude_patterns = ['output']
|
||||
|
||||
# The reST default role (used for this markup: `text`) to use for all
|
||||
# documents.
|
||||
#default_role = None
|
||||
|
||||
# If true, '()' will be appended to :func: etc. cross-reference text.
|
||||
#add_function_parentheses = True
|
||||
|
||||
# If true, the current module name will be prepended to all description
|
||||
# unit titles (such as .. function::).
|
||||
#add_module_names = True
|
||||
|
||||
# If true, sectionauthor and moduleauthor directives will be shown in the
|
||||
# output. They are ignored by default.
|
||||
#show_authors = False
|
||||
|
||||
# The name of the Pygments (syntax highlighting) style to use.
|
||||
pygments_style = 'sphinx'
|
||||
|
||||
# A list of ignored prefixes for module index sorting.
|
||||
#modindex_common_prefix = []
|
||||
|
||||
# If true, keep warnings as "system message" paragraphs in the built documents.
|
||||
#keep_warnings = False
|
||||
|
||||
# If true, `todo` and `todoList` produce output, else they produce nothing.
|
||||
todo_include_todos = False
|
||||
|
||||
primary_domain = 'C'
|
||||
highlight_language = 'C'
|
||||
|
||||
# -- Options for HTML output ----------------------------------------------
|
||||
|
||||
# The theme to use for HTML and HTML Help pages. See the documentation for
|
||||
# a list of builtin themes.
|
||||
|
||||
# The Read the Docs theme is available from
|
||||
# - https://github.com/snide/sphinx_rtd_theme
|
||||
# - https://pypi.python.org/pypi/sphinx_rtd_theme
|
||||
# - python-sphinx-rtd-theme package (on Debian)
|
||||
try:
|
||||
import sphinx_rtd_theme
|
||||
html_theme = 'sphinx_rtd_theme'
|
||||
html_theme_path = [sphinx_rtd_theme.get_html_theme_path()]
|
||||
except ImportError:
|
||||
sys.stderr.write('Warning: The Sphinx \'sphinx_rtd_theme\' HTML theme was not found. Make sure you have the theme installed to produce pretty HTML output. Falling back to the default theme.\n')
|
||||
|
||||
# Theme options are theme-specific and customize the look and feel of a theme
|
||||
# further. For a list of options available for each theme, see the
|
||||
# documentation.
|
||||
#html_theme_options = {}
|
||||
|
||||
# Add any paths that contain custom themes here, relative to this directory.
|
||||
#html_theme_path = []
|
||||
|
||||
# The name for this set of Sphinx documents. If None, it defaults to
|
||||
# "<project> v<release> documentation".
|
||||
#html_title = None
|
||||
|
||||
# A shorter title for the navigation bar. Default is the same as html_title.
|
||||
#html_short_title = None
|
||||
|
||||
# The name of an image file (relative to this directory) to place at the top
|
||||
# of the sidebar.
|
||||
#html_logo = None
|
||||
|
||||
# The name of an image file (within the static path) to use as favicon of the
|
||||
# docs. This file should be a Windows icon file (.ico) being 16x16 or 32x32
|
||||
# pixels large.
|
||||
#html_favicon = None
|
||||
|
||||
# Add any paths that contain custom static files (such as style sheets) here,
|
||||
# relative to this directory. They are copied after the builtin static files,
|
||||
# so a file named "default.css" will overwrite the builtin "default.css".
|
||||
#html_static_path = ['_static']
|
||||
|
||||
# Add any extra paths that contain custom files (such as robots.txt or
|
||||
# .htaccess) here, relative to this directory. These files are copied
|
||||
# directly to the root of the documentation.
|
||||
#html_extra_path = []
|
||||
|
||||
# If not '', a 'Last updated on:' timestamp is inserted at every page bottom,
|
||||
# using the given strftime format.
|
||||
#html_last_updated_fmt = '%b %d, %Y'
|
||||
|
||||
# If true, SmartyPants will be used to convert quotes and dashes to
|
||||
# typographically correct entities.
|
||||
#html_use_smartypants = True
|
||||
|
||||
# Custom sidebar templates, maps document names to template names.
|
||||
#html_sidebars = {}
|
||||
|
||||
# Additional templates that should be rendered to pages, maps page names to
|
||||
# template names.
|
||||
#html_additional_pages = {}
|
||||
|
||||
# If false, no module index is generated.
|
||||
#html_domain_indices = True
|
||||
|
||||
# If false, no index is generated.
|
||||
#html_use_index = True
|
||||
|
||||
# If true, the index is split into individual pages for each letter.
|
||||
#html_split_index = False
|
||||
|
||||
# If true, links to the reST sources are added to the pages.
|
||||
#html_show_sourcelink = True
|
||||
|
||||
# If true, "Created using Sphinx" is shown in the HTML footer. Default is True.
|
||||
#html_show_sphinx = True
|
||||
|
||||
# If true, "(C) Copyright ..." is shown in the HTML footer. Default is True.
|
||||
#html_show_copyright = True
|
||||
|
||||
# If true, an OpenSearch description file will be output, and all pages will
|
||||
# contain a <link> tag referring to it. The value of this option must be the
|
||||
# base URL from which the finished HTML is served.
|
||||
#html_use_opensearch = ''
|
||||
|
||||
# This is the file name suffix for HTML files (e.g. ".xhtml").
|
||||
#html_file_suffix = None
|
||||
|
||||
# Language to be used for generating the HTML full-text search index.
|
||||
# Sphinx supports the following languages:
|
||||
# 'da', 'de', 'en', 'es', 'fi', 'fr', 'h', 'it', 'ja'
|
||||
# 'nl', 'no', 'pt', 'ro', 'r', 'sv', 'tr'
|
||||
#html_search_language = 'en'
|
||||
|
||||
# A dictionary with options for the search language support, empty by default.
|
||||
# Now only 'ja' uses this config value
|
||||
#html_search_options = {'type': 'default'}
|
||||
|
||||
# The name of a javascript file (relative to the configuration directory) that
|
||||
# implements a search results scorer. If empty, the default will be used.
|
||||
#html_search_scorer = 'scorer.js'
|
||||
|
||||
# Output file base name for HTML help builder.
|
||||
htmlhelp_basename = 'TheLinuxKerneldoc'
|
||||
|
||||
# -- Options for LaTeX output ---------------------------------------------
|
||||
|
||||
latex_elements = {
|
||||
# The paper size ('letterpaper' or 'a4paper').
|
||||
#'papersize': 'letterpaper',
|
||||
|
||||
# The font size ('10pt', '11pt' or '12pt').
|
||||
#'pointsize': '10pt',
|
||||
|
||||
# Additional stuff for the LaTeX preamble.
|
||||
#'preamble': '',
|
||||
|
||||
# Latex figure (float) alignment
|
||||
#'figure_align': 'htbp',
|
||||
}
|
||||
|
||||
# Grouping the document tree into LaTeX files. List of tuples
|
||||
# (source start file, target name, title,
|
||||
# author, documentclass [howto, manual, or own class]).
|
||||
latex_documents = [
|
||||
(master_doc, 'TheLinuxKernel.tex', 'The Linux Kernel Documentation',
|
||||
'The kernel development community', 'manual'),
|
||||
]
|
||||
|
||||
# The name of an image file (relative to this directory) to place at the top of
|
||||
# the title page.
|
||||
#latex_logo = None
|
||||
|
||||
# For "manual" documents, if this is true, then toplevel headings are parts,
|
||||
# not chapters.
|
||||
#latex_use_parts = False
|
||||
|
||||
# If true, show page references after internal links.
|
||||
#latex_show_pagerefs = False
|
||||
|
||||
# If true, show URL addresses after external links.
|
||||
#latex_show_urls = False
|
||||
|
||||
# Documents to append as an appendix to all manuals.
|
||||
#latex_appendices = []
|
||||
|
||||
# If false, no module index is generated.
|
||||
#latex_domain_indices = True
|
||||
|
||||
|
||||
# -- Options for manual page output ---------------------------------------
|
||||
|
||||
# One entry per manual page. List of tuples
|
||||
# (source start file, name, description, authors, manual section).
|
||||
man_pages = [
|
||||
(master_doc, 'thelinuxkernel', 'The Linux Kernel Documentation',
|
||||
[author], 1)
|
||||
]
|
||||
|
||||
# If true, show URL addresses after external links.
|
||||
#man_show_urls = False
|
||||
|
||||
|
||||
# -- Options for Texinfo output -------------------------------------------
|
||||
|
||||
# Grouping the document tree into Texinfo files. List of tuples
|
||||
# (source start file, target name, title, author,
|
||||
# dir menu entry, description, category)
|
||||
texinfo_documents = [
|
||||
(master_doc, 'TheLinuxKernel', 'The Linux Kernel Documentation',
|
||||
author, 'TheLinuxKernel', 'One line description of project.',
|
||||
'Miscellaneous'),
|
||||
]
|
||||
|
||||
# Documents to append as an appendix to all manuals.
|
||||
#texinfo_appendices = []
|
||||
|
||||
# If false, no module index is generated.
|
||||
#texinfo_domain_indices = True
|
||||
|
||||
# How to display URL addresses: 'footnote', 'no', or 'inline'.
|
||||
#texinfo_show_urls = 'footnote'
|
||||
|
||||
# If true, do not generate a @detailmenu in the "Top" node's menu.
|
||||
#texinfo_no_detailmenu = False
|
||||
|
||||
|
||||
# -- Options for Epub output ----------------------------------------------
|
||||
|
||||
# Bibliographic Dublin Core info.
|
||||
epub_title = project
|
||||
epub_author = author
|
||||
epub_publisher = author
|
||||
epub_copyright = copyright
|
||||
|
||||
# The basename for the epub file. It defaults to the project name.
|
||||
#epub_basename = project
|
||||
|
||||
# The HTML theme for the epub output. Since the default themes are not
|
||||
# optimized for small screen space, using the same theme for HTML and epub
|
||||
# output is usually not wise. This defaults to 'epub', a theme designed to save
|
||||
# visual space.
|
||||
#epub_theme = 'epub'
|
||||
|
||||
# The language of the text. It defaults to the language option
|
||||
# or 'en' if the language is not set.
|
||||
#epub_language = ''
|
||||
|
||||
# The scheme of the identifier. Typical schemes are ISBN or URL.
|
||||
#epub_scheme = ''
|
||||
|
||||
# The unique identifier of the text. This can be a ISBN number
|
||||
# or the project homepage.
|
||||
#epub_identifier = ''
|
||||
|
||||
# A unique identification for the text.
|
||||
#epub_uid = ''
|
||||
|
||||
# A tuple containing the cover image and cover page html template filenames.
|
||||
#epub_cover = ()
|
||||
|
||||
# A sequence of (type, uri, title) tuples for the guide element of content.opf.
|
||||
#epub_guide = ()
|
||||
|
||||
# HTML files that should be inserted before the pages created by sphinx.
|
||||
# The format is a list of tuples containing the path and title.
|
||||
#epub_pre_files = []
|
||||
|
||||
# HTML files that should be inserted after the pages created by sphinx.
|
||||
# The format is a list of tuples containing the path and title.
|
||||
#epub_post_files = []
|
||||
|
||||
# A list of files that should not be packed into the epub file.
|
||||
epub_exclude_files = ['search.html']
|
||||
|
||||
# The depth of the table of contents in toc.ncx.
|
||||
#epub_tocdepth = 3
|
||||
|
||||
# Allow duplicate toc entries.
|
||||
#epub_tocdup = True
|
||||
|
||||
# Choose between 'default' and 'includehidden'.
|
||||
#epub_tocscope = 'default'
|
||||
|
||||
# Fix unsupported image types using the Pillow.
|
||||
#epub_fix_images = False
|
||||
|
||||
# Scale large images.
|
||||
#epub_max_image_width = 0
|
||||
|
||||
# How to display URL addresses: 'footnote', 'no', or 'inline'.
|
||||
#epub_show_urls = 'inline'
|
||||
|
||||
# If false, no index is generated.
|
||||
#epub_use_index = True
|
||||
|
||||
#=======
|
||||
# rst2pdf
|
||||
#
|
||||
# Grouping the document tree into PDF files. List of tuples
|
||||
# (source start file, target name, title, author, options).
|
||||
#
|
||||
# See the Sphinx chapter of http://ralsina.me/static/manual.pdf
|
||||
#
|
||||
# FIXME: Do not add the index file here; the result will be too big. Adding
|
||||
# multiple PDF files here actually tries to get the cross-referencing right
|
||||
# *between* PDF files.
|
||||
pdf_documents = [
|
||||
('index', u'Kernel', u'Kernel', u'J. Random Bozo'),
|
||||
]
|
||||
|
||||
# kernel-doc extension configuration for running Sphinx directly (e.g. by Read
|
||||
# the Docs). In a normal build, these are supplied from the Makefile via command
|
||||
# line arguments.
|
||||
kerneldoc_bin = '../scripts/kernel-doc'
|
||||
kerneldoc_srctree = '..'
|
||||
|
|
@ -346,7 +346,7 @@ which have not been so documented, there is no harm in adding kerneldoc
|
|||
comments for the future; indeed, this can be a useful activity for
|
||||
beginning kernel developers. The format of these comments, along with some
|
||||
information on how to create kerneldoc templates can be found in the file
|
||||
Documentation/kernel-doc-nano-HOWTO.txt.
|
||||
Documentation/kernel-documentation.rst.
|
||||
|
||||
Anybody who reads through a significant amount of existing kernel code will
|
||||
note that, often, comments are most notable by their absence. Once again,
|
||||
|
|
|
|||
|
|
@ -323,7 +323,7 @@ supported.
|
|||
* device_resume
|
||||
- Resumes a transfer on the channel
|
||||
- This command should operate synchronously on the channel,
|
||||
pausing right away the work of the given channel
|
||||
resuming right away the work of the given channel
|
||||
|
||||
* device_terminate_all
|
||||
- Aborts all the pending and ongoing transfers on the channel
|
||||
|
|
|
|||
|
|
@ -5,12 +5,12 @@ This document will describe OCFS2 online file check feature.
|
|||
|
||||
Introduction
|
||||
============
|
||||
OCFS2 is often used in high-availaibility systems. However, OCFS2 usually
|
||||
OCFS2 is often used in high-availability systems. However, OCFS2 usually
|
||||
converts the filesystem to read-only when encounters an error. This may not be
|
||||
necessary, since turning the filesystem read-only would affect other running
|
||||
processes as well, decreasing availability.
|
||||
Then, a mount option (errors=continue) is introduced, which would return the
|
||||
-EIO errno to the calling process and terminate furhter processing so that the
|
||||
-EIO errno to the calling process and terminate further processing so that the
|
||||
filesystem is not corrupted further. The filesystem is not converted to
|
||||
read-only, and the problematic file's inode number is reported in the kernel
|
||||
log. The user can try to check/fix this file via online filecheck feature.
|
||||
|
|
@ -44,7 +44,7 @@ There is a sysfs directory for each OCFS2 file system mounting:
|
|||
|
||||
/sys/fs/ocfs2/<devname>/filecheck
|
||||
|
||||
Here, <devname> indicates the name of OCFS2 volumn device which has been already
|
||||
Here, <devname> indicates the name of OCFS2 volume device which has been already
|
||||
mounted. The file above would accept inode numbers. This could be used to
|
||||
communicate with kernel space, tell which file(inode number) will be checked or
|
||||
fixed. Currently, three operations are supported, which includes checking
|
||||
|
|
@ -76,14 +76,14 @@ The output is like this:
|
|||
This time, the <ERROR> column indicates whether this fix is successful or not.
|
||||
|
||||
3. The record cache is used to store the history of check/fix results. It's
|
||||
defalut size is 10, and can be adjust between the range of 10 ~ 100. You can
|
||||
default size is 10, and can be adjust between the range of 10 ~ 100. You can
|
||||
adjust the size like this:
|
||||
|
||||
# echo "<size>" > /sys/fs/ocfs2/<devname>/filecheck/set
|
||||
|
||||
Fixing stuff
|
||||
============
|
||||
On receivng the inode, the filesystem would read the inode and the
|
||||
On receiving the inode, the filesystem would read the inode and the
|
||||
file metadata. In case of errors, the filesystem would fix the errors
|
||||
and report the problems it fixed in the kernel log. As a precautionary measure,
|
||||
the inode must first be checked for errors before performing a final fix.
|
||||
|
|
|
|||
22
Documentation/index.rst
Normal file
22
Documentation/index.rst
Normal file
|
|
@ -0,0 +1,22 @@
|
|||
.. The Linux Kernel documentation master file, created by
|
||||
sphinx-quickstart on Fri Feb 12 13:51:46 2016.
|
||||
You can adapt this file completely to your liking, but it should at least
|
||||
contain the root `toctree` directive.
|
||||
|
||||
Welcome to The Linux Kernel's documentation!
|
||||
============================================
|
||||
|
||||
Nothing for you to see here *yet*. Please move along.
|
||||
|
||||
Contents:
|
||||
|
||||
.. toctree::
|
||||
:maxdepth: 2
|
||||
|
||||
kernel-documentation
|
||||
|
||||
Indices and tables
|
||||
==================
|
||||
|
||||
* :ref:`genindex`
|
||||
* :ref:`search`
|
||||
|
|
@ -1,3 +1,6 @@
|
|||
NOTE: this document is outdated and will eventually be removed. See
|
||||
Documentation/kernel-documentation.rst for current information.
|
||||
|
||||
kernel-doc nano-HOWTO
|
||||
=====================
|
||||
|
||||
|
|
|
|||
654
Documentation/kernel-documentation.rst
Normal file
654
Documentation/kernel-documentation.rst
Normal file
|
|
@ -0,0 +1,654 @@
|
|||
==========================
|
||||
Linux Kernel Documentation
|
||||
==========================
|
||||
|
||||
Introduction
|
||||
============
|
||||
|
||||
The Linux kernel uses `Sphinx`_ to generate pretty documentation from
|
||||
`reStructuredText`_ files under ``Documentation``. To build the documentation in
|
||||
HTML or PDF formats, use ``make htmldocs`` or ``make pdfdocs``. The generated
|
||||
documentation is placed in ``Documentation/output``.
|
||||
|
||||
.. _Sphinx: http://www.sphinx-doc.org/
|
||||
.. _reStructuredText: http://docutils.sourceforge.net/rst.html
|
||||
|
||||
The reStructuredText files may contain directives to include structured
|
||||
documentation comments, or kernel-doc comments, from source files. Usually these
|
||||
are used to describe the functions and types and design of the code. The
|
||||
kernel-doc comments have some special structure and formatting, but beyond that
|
||||
they are also treated as reStructuredText.
|
||||
|
||||
There is also the deprecated DocBook toolchain to generate documentation from
|
||||
DocBook XML template files under ``Documentation/DocBook``. The DocBook files
|
||||
are to be converted to reStructuredText, and the toolchain is slated to be
|
||||
removed.
|
||||
|
||||
Finally, there are thousands of plain text documentation files scattered around
|
||||
``Documentation``. Some of these will likely be converted to reStructuredText
|
||||
over time, but the bulk of them will remain in plain text.
|
||||
|
||||
Sphinx Build
|
||||
============
|
||||
|
||||
The usual way to generate the documentation is to run ``make htmldocs`` or
|
||||
``make pdfdocs``. There are also other formats available, see the documentation
|
||||
section of ``make help``. The generated documentation is placed in
|
||||
format-specific subdirectories under ``Documentation/output``.
|
||||
|
||||
To generate documentation, Sphinx (``sphinx-build``) must obviously be
|
||||
installed. For prettier HTML output, the Read the Docs Sphinx theme
|
||||
(``sphinx_rtd_theme``) is used if available. For PDF output, ``rst2pdf`` is also
|
||||
needed. All of these are widely available and packaged in distributions.
|
||||
|
||||
To pass extra options to Sphinx, you can use the ``SPHINXOPTS`` make
|
||||
variable. For example, use ``make SPHINXOPTS=-v htmldocs`` to get more verbose
|
||||
output.
|
||||
|
||||
To remove the generated documentation, run ``make cleandocs``.
|
||||
|
||||
Writing Documentation
|
||||
=====================
|
||||
|
||||
Adding new documentation can be as simple as:
|
||||
|
||||
1. Add a new ``.rst`` file somewhere under ``Documentation``.
|
||||
2. Refer to it from the Sphinx main `TOC tree`_ in ``Documentation/index.rst``.
|
||||
|
||||
.. _TOC tree: http://www.sphinx-doc.org/en/stable/markup/toctree.html
|
||||
|
||||
This is usually good enough for simple documentation (like the one you're
|
||||
reading right now), but for larger documents it may be advisable to create a
|
||||
subdirectory (or use an existing one). For example, the graphics subsystem
|
||||
documentation is under ``Documentation/gpu``, split to several ``.rst`` files,
|
||||
and has a separate ``index.rst`` (with a ``toctree`` of its own) referenced from
|
||||
the main index.
|
||||
|
||||
See the documentation for `Sphinx`_ and `reStructuredText`_ on what you can do
|
||||
with them. In particular, the Sphinx `reStructuredText Primer`_ is a good place
|
||||
to get started with reStructuredText. There are also some `Sphinx specific
|
||||
markup constructs`_.
|
||||
|
||||
.. _reStructuredText Primer: http://www.sphinx-doc.org/en/stable/rest.html
|
||||
.. _Sphinx specific markup constructs: http://www.sphinx-doc.org/en/stable/markup/index.html
|
||||
|
||||
Specific guidelines for the kernel documentation
|
||||
------------------------------------------------
|
||||
|
||||
Here are some specific guidelines for the kernel documentation:
|
||||
|
||||
* Please don't go overboard with reStructuredText markup. Keep it simple.
|
||||
|
||||
* Please stick to this order of heading adornments:
|
||||
|
||||
1. ``=`` with overline for document title::
|
||||
|
||||
==============
|
||||
Document title
|
||||
==============
|
||||
|
||||
2. ``=`` for chapters::
|
||||
|
||||
Chapters
|
||||
========
|
||||
|
||||
3. ``-`` for sections::
|
||||
|
||||
Section
|
||||
-------
|
||||
|
||||
4. ``~`` for subsections::
|
||||
|
||||
Subsection
|
||||
~~~~~~~~~~
|
||||
|
||||
Although RST doesn't mandate a specific order ("Rather than imposing a fixed
|
||||
number and order of section title adornment styles, the order enforced will be
|
||||
the order as encountered."), having the higher levels the same overall makes
|
||||
it easier to follow the documents.
|
||||
|
||||
list tables
|
||||
-----------
|
||||
|
||||
We recommend the use of *list table* formats. The *list table* formats are
|
||||
double-stage lists. Compared to the ASCII-art they might not be as
|
||||
comfortable for
|
||||
readers of the text files. Their advantage is that they are easy to
|
||||
create or modify and that the diff of a modification is much more meaningful,
|
||||
because it is limited to the modified content.
|
||||
|
||||
The ``flat-table`` is a double-stage list similar to the ``list-table`` with
|
||||
some additional features:
|
||||
|
||||
* column-span: with the role ``cspan`` a cell can be extended through
|
||||
additional columns
|
||||
|
||||
* row-span: with the role ``rspan`` a cell can be extended through
|
||||
additional rows
|
||||
|
||||
* auto span rightmost cell of a table row over the missing cells on the right
|
||||
side of that table-row. With Option ``:fill-cells:`` this behavior can
|
||||
changed from *auto span* to *auto fill*, which automatically inserts (empty)
|
||||
cells instead of spanning the last cell.
|
||||
|
||||
options:
|
||||
|
||||
* ``:header-rows:`` [int] count of header rows
|
||||
* ``:stub-columns:`` [int] count of stub columns
|
||||
* ``:widths:`` [[int] [int] ... ] widths of columns
|
||||
* ``:fill-cells:`` instead of auto-spanning missing cells, insert missing cells
|
||||
|
||||
roles:
|
||||
|
||||
* ``:cspan:`` [int] additional columns (*morecols*)
|
||||
* ``:rspan:`` [int] additional rows (*morerows*)
|
||||
|
||||
The example below shows how to use this markup. The first level of the staged
|
||||
list is the *table-row*. In the *table-row* there is only one markup allowed,
|
||||
the list of the cells in this *table-row*. Exceptions are *comments* ( ``..`` )
|
||||
and *targets* (e.g. a ref to ``:ref:`last row <last row>``` / :ref:`last row
|
||||
<last row>`).
|
||||
|
||||
.. code-block:: rst
|
||||
|
||||
.. flat-table:: table title
|
||||
:widths: 2 1 1 3
|
||||
|
||||
* - head col 1
|
||||
- head col 2
|
||||
- head col 3
|
||||
- head col 4
|
||||
|
||||
* - column 1
|
||||
- field 1.1
|
||||
- field 1.2 with autospan
|
||||
|
||||
* - column 2
|
||||
- field 2.1
|
||||
- :rspan:`1` :cspan:`1` field 2.2 - 3.3
|
||||
|
||||
* .. _`last row`:
|
||||
|
||||
- column 3
|
||||
|
||||
Rendered as:
|
||||
|
||||
.. flat-table:: table title
|
||||
:widths: 2 1 1 3
|
||||
|
||||
* - head col 1
|
||||
- head col 2
|
||||
- head col 3
|
||||
- head col 4
|
||||
|
||||
* - column 1
|
||||
- field 1.1
|
||||
- field 1.2 with autospan
|
||||
|
||||
* - column 2
|
||||
- field 2.1
|
||||
- :rspan:`1` :cspan:`1` field 2.2 - 3.3
|
||||
|
||||
* .. _`last row`:
|
||||
|
||||
- column 3
|
||||
|
||||
|
||||
Including kernel-doc comments
|
||||
=============================
|
||||
|
||||
The Linux kernel source files may contain structured documentation comments, or
|
||||
kernel-doc comments to describe the functions and types and design of the
|
||||
code. The documentation comments may be included to any of the reStructuredText
|
||||
documents using a dedicated kernel-doc Sphinx directive extension.
|
||||
|
||||
The kernel-doc directive is of the format::
|
||||
|
||||
.. kernel-doc:: source
|
||||
:option:
|
||||
|
||||
The *source* is the path to a source file, relative to the kernel source
|
||||
tree. The following directive options are supported:
|
||||
|
||||
export: *[source-pattern ...]*
|
||||
Include documentation for all functions in *source* that have been exported
|
||||
using ``EXPORT_SYMBOL`` or ``EXPORT_SYMBOL_GPL`` either in *source* or in any
|
||||
of the files specified by *source-pattern*.
|
||||
|
||||
The *source-pattern* is useful when the kernel-doc comments have been placed
|
||||
in header files, while ``EXPORT_SYMBOL`` and ``EXPORT_SYMBOL_GPL`` are next to
|
||||
the function definitions.
|
||||
|
||||
Examples::
|
||||
|
||||
.. kernel-doc:: lib/bitmap.c
|
||||
:export:
|
||||
|
||||
.. kernel-doc:: include/net/mac80211.h
|
||||
:export: net/mac80211/*.c
|
||||
|
||||
internal: *[source-pattern ...]*
|
||||
Include documentation for all functions and types in *source* that have
|
||||
**not** been exported using ``EXPORT_SYMBOL`` or ``EXPORT_SYMBOL_GPL`` either
|
||||
in *source* or in any of the files specified by *source-pattern*.
|
||||
|
||||
Example::
|
||||
|
||||
.. kernel-doc:: drivers/gpu/drm/i915/intel_audio.c
|
||||
:internal:
|
||||
|
||||
doc: *title*
|
||||
Include documentation for the ``DOC:`` paragraph identified by *title* in
|
||||
*source*. Spaces are allowed in *title*; do not quote the *title*. The *title*
|
||||
is only used as an identifier for the paragraph, and is not included in the
|
||||
output. Please make sure to have an appropriate heading in the enclosing
|
||||
reStructuredText document.
|
||||
|
||||
Example::
|
||||
|
||||
.. kernel-doc:: drivers/gpu/drm/i915/intel_audio.c
|
||||
:doc: High Definition Audio over HDMI and Display Port
|
||||
|
||||
functions: *function* *[...]*
|
||||
Include documentation for each *function* in *source*.
|
||||
|
||||
Example::
|
||||
|
||||
.. kernel-doc:: lib/bitmap.c
|
||||
:functions: bitmap_parselist bitmap_parselist_user
|
||||
|
||||
Without options, the kernel-doc directive includes all documentation comments
|
||||
from the source file.
|
||||
|
||||
The kernel-doc extension is included in the kernel source tree, at
|
||||
``Documentation/sphinx/kernel-doc.py``. Internally, it uses the
|
||||
``scripts/kernel-doc`` script to extract the documentation comments from the
|
||||
source.
|
||||
|
||||
Writing kernel-doc comments
|
||||
===========================
|
||||
|
||||
In order to provide embedded, "C" friendly, easy to maintain, but consistent and
|
||||
extractable overview, function and type documentation, the Linux kernel has
|
||||
adopted a consistent style for documentation comments. The format for this
|
||||
documentation is called the kernel-doc format, described below. This style
|
||||
embeds the documentation within the source files, using a few simple conventions
|
||||
for adding documentation paragraphs and documenting functions and their
|
||||
parameters, structures and unions and their members, enumerations, and typedefs.
|
||||
|
||||
.. note:: The kernel-doc format is deceptively similar to gtk-doc or Doxygen,
|
||||
yet distinctively different, for historical reasons. The kernel source
|
||||
contains tens of thousands of kernel-doc comments. Please stick to the style
|
||||
described here.
|
||||
|
||||
The ``scripts/kernel-doc`` script is used by the Sphinx kernel-doc extension in
|
||||
the documentation build to extract this embedded documentation into the various
|
||||
HTML, PDF, and other format documents.
|
||||
|
||||
In order to provide good documentation of kernel functions and data structures,
|
||||
please use the following conventions to format your kernel-doc comments in the
|
||||
Linux kernel source.
|
||||
|
||||
How to format kernel-doc comments
|
||||
---------------------------------
|
||||
|
||||
The opening comment mark ``/**`` is reserved for kernel-doc comments. Only
|
||||
comments so marked will be considered by the ``kernel-doc`` tool. Use it only
|
||||
for comment blocks that contain kernel-doc formatted comments. The usual ``*/``
|
||||
should be used as the closing comment marker. The lines in between should be
|
||||
prefixed by `` * `` (space star space).
|
||||
|
||||
The function and type kernel-doc comments should be placed just before the
|
||||
function or type being described. The overview kernel-doc comments may be freely
|
||||
placed at the top indentation level.
|
||||
|
||||
Example kernel-doc function comment::
|
||||
|
||||
/**
|
||||
* foobar() - Brief description of foobar.
|
||||
* @arg: Description of argument of foobar.
|
||||
*
|
||||
* Longer description of foobar.
|
||||
*
|
||||
* Return: Description of return value of foobar.
|
||||
*/
|
||||
int foobar(int arg)
|
||||
|
||||
The format is similar for documentation for structures, enums, paragraphs,
|
||||
etc. See the sections below for details.
|
||||
|
||||
The kernel-doc structure is extracted from the comments, and proper `Sphinx C
|
||||
Domain`_ function and type descriptions with anchors are generated for them. The
|
||||
descriptions are filtered for special kernel-doc highlights and
|
||||
cross-references. See below for details.
|
||||
|
||||
.. _Sphinx C Domain: http://www.sphinx-doc.org/en/stable/domains.html
|
||||
|
||||
Highlights and cross-references
|
||||
-------------------------------
|
||||
|
||||
The following special patterns are recognized in the kernel-doc comment
|
||||
descriptive text and converted to proper reStructuredText markup and `Sphinx C
|
||||
Domain`_ references.
|
||||
|
||||
.. attention:: The below are **only** recognized within kernel-doc comments,
|
||||
**not** within normal reStructuredText documents.
|
||||
|
||||
``funcname()``
|
||||
Function reference.
|
||||
|
||||
``@parameter``
|
||||
Name of a function parameter. (No cross-referencing, just formatting.)
|
||||
|
||||
``%CONST``
|
||||
Name of a constant. (No cross-referencing, just formatting.)
|
||||
|
||||
``$ENVVAR``
|
||||
Name of an environment variable. (No cross-referencing, just formatting.)
|
||||
|
||||
``&struct name``
|
||||
Structure reference.
|
||||
|
||||
``&enum name``
|
||||
Enum reference.
|
||||
|
||||
``&typedef name``
|
||||
Typedef reference.
|
||||
|
||||
``&struct_name->member`` or ``&struct_name.member``
|
||||
Structure or union member reference. The cross-reference will be to the struct
|
||||
or union definition, not the member directly.
|
||||
|
||||
``&name``
|
||||
A generic type reference. Prefer using the full reference described above
|
||||
instead. This is mostly for legacy comments.
|
||||
|
||||
Cross-referencing from reStructuredText
|
||||
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
|
||||
|
||||
.. highlight:: none
|
||||
|
||||
To cross-reference the functions and types defined in the kernel-doc comments
|
||||
from reStructuredText documents, please use the `Sphinx C Domain`_
|
||||
references. For example::
|
||||
|
||||
See function :c:func:`foo` and struct/union/enum/typedef :c:type:`bar`.
|
||||
|
||||
While the type reference works with just the type name, without the
|
||||
struct/union/enum/typedef part in front, you may want to use::
|
||||
|
||||
See :c:type:`struct foo <foo>`.
|
||||
See :c:type:`union bar <bar>`.
|
||||
See :c:type:`enum baz <baz>`.
|
||||
See :c:type:`typedef meh <meh>`.
|
||||
|
||||
This will produce prettier links, and is in line with how kernel-doc does the
|
||||
cross-references.
|
||||
|
||||
For further details, please refer to the `Sphinx C Domain`_ documentation.
|
||||
|
||||
Function documentation
|
||||
----------------------
|
||||
|
||||
.. highlight:: c
|
||||
|
||||
The general format of a function and function-like macro kernel-doc comment is::
|
||||
|
||||
/**
|
||||
* function_name() - Brief description of function.
|
||||
* @arg1: Describe the first argument.
|
||||
* @arg2: Describe the second argument.
|
||||
* One can provide multiple line descriptions
|
||||
* for arguments.
|
||||
*
|
||||
* A longer description, with more discussion of the function function_name()
|
||||
* that might be useful to those using or modifying it. Begins with an
|
||||
* empty comment line, and may include additional embedded empty
|
||||
* comment lines.
|
||||
*
|
||||
* The longer description may have multiple paragraphs.
|
||||
*
|
||||
* Return: Describe the return value of foobar.
|
||||
*
|
||||
* The return value description can also have multiple paragraphs, and should
|
||||
* be placed at the end of the comment block.
|
||||
*/
|
||||
|
||||
The brief description following the function name may span multiple lines, and
|
||||
ends with an ``@argument:`` description, a blank comment line, or the end of the
|
||||
comment block.
|
||||
|
||||
The kernel-doc function comments describe each parameter to the function, in
|
||||
order, with the ``@argument:`` descriptions. The ``@argument:`` descriptions
|
||||
must begin on the very next line following the opening brief function
|
||||
description line, with no intervening blank comment lines. The ``@argument:``
|
||||
descriptions may span multiple lines. The continuation lines may contain
|
||||
indentation. If a function parameter is ``...`` (varargs), it should be listed
|
||||
in kernel-doc notation as: ``@...:``.
|
||||
|
||||
The return value, if any, should be described in a dedicated section at the end
|
||||
of the comment starting with "Return:".
|
||||
|
||||
Structure, union, and enumeration documentation
|
||||
-----------------------------------------------
|
||||
|
||||
The general format of a struct, union, and enum kernel-doc comment is::
|
||||
|
||||
/**
|
||||
* struct struct_name - Brief description.
|
||||
* @member_name: Description of member member_name.
|
||||
*
|
||||
* Description of the structure.
|
||||
*/
|
||||
|
||||
Below, "struct" is used to mean structs, unions and enums, and "member" is used
|
||||
to mean struct and union members as well as enumerations in an enum.
|
||||
|
||||
The brief description following the structure name may span multiple lines, and
|
||||
ends with a ``@member:`` description, a blank comment line, or the end of the
|
||||
comment block.
|
||||
|
||||
The kernel-doc data structure comments describe each member of the structure, in
|
||||
order, with the ``@member:`` descriptions. The ``@member:`` descriptions must
|
||||
begin on the very next line following the opening brief function description
|
||||
line, with no intervening blank comment lines. The ``@member:`` descriptions may
|
||||
span multiple lines. The continuation lines may contain indentation.
|
||||
|
||||
In-line member documentation comments
|
||||
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
|
||||
|
||||
The structure members may also be documented in-line within the definition::
|
||||
|
||||
/**
|
||||
* struct foo - Brief description.
|
||||
* @foo: The Foo member.
|
||||
*/
|
||||
struct foo {
|
||||
int foo;
|
||||
/**
|
||||
* @bar: The Bar member.
|
||||
*/
|
||||
int bar;
|
||||
/**
|
||||
* @baz: The Baz member.
|
||||
*
|
||||
* Here, the member description may contain several paragraphs.
|
||||
*/
|
||||
int baz;
|
||||
}
|
||||
|
||||
Private members
|
||||
~~~~~~~~~~~~~~~
|
||||
|
||||
Inside a struct description, you can use the "private:" and "public:" comment
|
||||
tags. Structure fields that are inside a "private:" area are not listed in the
|
||||
generated output documentation. The "private:" and "public:" tags must begin
|
||||
immediately following a ``/*`` comment marker. They may optionally include
|
||||
comments between the ``:`` and the ending ``*/`` marker.
|
||||
|
||||
Example::
|
||||
|
||||
/**
|
||||
* struct my_struct - short description
|
||||
* @a: first member
|
||||
* @b: second member
|
||||
*
|
||||
* Longer description
|
||||
*/
|
||||
struct my_struct {
|
||||
int a;
|
||||
int b;
|
||||
/* private: internal use only */
|
||||
int c;
|
||||
};
|
||||
|
||||
|
||||
Typedef documentation
|
||||
---------------------
|
||||
|
||||
The general format of a typedef kernel-doc comment is::
|
||||
|
||||
/**
|
||||
* typedef type_name - Brief description.
|
||||
*
|
||||
* Description of the type.
|
||||
*/
|
||||
|
||||
Overview documentation comments
|
||||
-------------------------------
|
||||
|
||||
To facilitate having source code and comments close together, you can include
|
||||
kernel-doc documentation blocks that are free-form comments instead of being
|
||||
kernel-doc for functions, structures, unions, enums, or typedefs. This could be
|
||||
used for something like a theory of operation for a driver or library code, for
|
||||
example.
|
||||
|
||||
This is done by using a ``DOC:`` section keyword with a section title.
|
||||
|
||||
The general format of an overview or high-level documentation comment is::
|
||||
|
||||
/**
|
||||
* DOC: Theory of Operation
|
||||
*
|
||||
* The whizbang foobar is a dilly of a gizmo. It can do whatever you
|
||||
* want it to do, at any time. It reads your mind. Here's how it works.
|
||||
*
|
||||
* foo bar splat
|
||||
*
|
||||
* The only drawback to this gizmo is that is can sometimes damage
|
||||
* hardware, software, or its subject(s).
|
||||
*/
|
||||
|
||||
The title following ``DOC:`` acts as a heading within the source file, but also
|
||||
as an identifier for extracting the documentation comment. Thus, the title must
|
||||
be unique within the file.
|
||||
|
||||
Recommendations
|
||||
---------------
|
||||
|
||||
We definitely need kernel-doc formatted documentation for functions that are
|
||||
exported to loadable modules using ``EXPORT_SYMBOL`` or ``EXPORT_SYMBOL_GPL``.
|
||||
|
||||
We also look to provide kernel-doc formatted documentation for functions
|
||||
externally visible to other kernel files (not marked "static").
|
||||
|
||||
We also recommend providing kernel-doc formatted documentation for private (file
|
||||
"static") routines, for consistency of kernel source code layout. But this is
|
||||
lower priority and at the discretion of the MAINTAINER of that kernel source
|
||||
file.
|
||||
|
||||
Data structures visible in kernel include files should also be documented using
|
||||
kernel-doc formatted comments.
|
||||
|
||||
DocBook XML [DEPRECATED]
|
||||
========================
|
||||
|
||||
.. attention::
|
||||
|
||||
This section describes the deprecated DocBook XML toolchain. Please do not
|
||||
create new DocBook XML template files. Please consider converting existing
|
||||
DocBook XML templates files to Sphinx/reStructuredText.
|
||||
|
||||
Converting DocBook to Sphinx
|
||||
----------------------------
|
||||
|
||||
.. highlight:: none
|
||||
|
||||
Over time, we expect all of the documents under ``Documentation/DocBook`` to be
|
||||
converted to Sphinx and reStructuredText. For most DocBook XML documents, a good
|
||||
enough solution is to use the simple ``Documentation/sphinx/tmplcvt`` script,
|
||||
which uses ``pandoc`` under the hood. For example::
|
||||
|
||||
$ cd Documentation/sphinx
|
||||
$ ./tmplcvt ../DocBook/in.tmpl ../out.rst
|
||||
|
||||
Then edit the resulting rst files to fix any remaining issues, and add the
|
||||
document in the ``toctree`` in ``Documentation/index.rst``.
|
||||
|
||||
Components of the kernel-doc system
|
||||
-----------------------------------
|
||||
|
||||
Many places in the source tree have extractable documentation in the form of
|
||||
block comments above functions. The components of this system are:
|
||||
|
||||
- ``scripts/kernel-doc``
|
||||
|
||||
This is a perl script that hunts for the block comments and can mark them up
|
||||
directly into reStructuredText, DocBook, man, text, and HTML. (No, not
|
||||
texinfo.)
|
||||
|
||||
- ``Documentation/DocBook/*.tmpl``
|
||||
|
||||
These are XML template files, which are normal XML files with special
|
||||
place-holders for where the extracted documentation should go.
|
||||
|
||||
- ``scripts/docproc.c``
|
||||
|
||||
This is a program for converting XML template files into XML files. When a
|
||||
file is referenced it is searched for symbols exported (EXPORT_SYMBOL), to be
|
||||
able to distinguish between internal and external functions.
|
||||
|
||||
It invokes kernel-doc, giving it the list of functions that are to be
|
||||
documented.
|
||||
|
||||
Additionally it is used to scan the XML template files to locate all the files
|
||||
referenced herein. This is used to generate dependency information as used by
|
||||
make.
|
||||
|
||||
- ``Makefile``
|
||||
|
||||
The targets 'xmldocs', 'psdocs', 'pdfdocs', and 'htmldocs' are used to build
|
||||
DocBook XML files, PostScript files, PDF files, and html files in
|
||||
Documentation/DocBook. The older target 'sgmldocs' is equivalent to 'xmldocs'.
|
||||
|
||||
- ``Documentation/DocBook/Makefile``
|
||||
|
||||
This is where C files are associated with SGML templates.
|
||||
|
||||
How to use kernel-doc comments in DocBook XML template files
|
||||
------------------------------------------------------------
|
||||
|
||||
DocBook XML template files (\*.tmpl) are like normal XML files, except that they
|
||||
can contain escape sequences where extracted documentation should be inserted.
|
||||
|
||||
``!E<filename>`` is replaced by the documentation, in ``<filename>``, for
|
||||
functions that are exported using ``EXPORT_SYMBOL``: the function list is
|
||||
collected from files listed in ``Documentation/DocBook/Makefile``.
|
||||
|
||||
``!I<filename>`` is replaced by the documentation for functions that are **not**
|
||||
exported using ``EXPORT_SYMBOL``.
|
||||
|
||||
``!D<filename>`` is used to name additional files to search for functions
|
||||
exported using ``EXPORT_SYMBOL``.
|
||||
|
||||
``!F<filename> <function [functions...]>`` is replaced by the documentation, in
|
||||
``<filename>``, for the functions listed.
|
||||
|
||||
``!P<filename> <section title>`` is replaced by the contents of the ``DOC:``
|
||||
section titled ``<section title>`` from ``<filename>``. Spaces are allowed in
|
||||
``<section title>``; do not quote the ``<section title>``.
|
||||
|
||||
``!C<filename>`` is replaced by nothing, but makes the tools check that all DOC:
|
||||
sections and documented functions, symbols, etc. are used. This makes sense to
|
||||
use when you use ``!F`` or ``!P`` only and want to verify that all documentation
|
||||
is included.
|
||||
|
|
@ -3996,8 +3996,9 @@ bytes respectively. Such letter suffixes can also be entirely omitted.
|
|||
|
||||
trace_event=[event-list]
|
||||
[FTRACE] Set and start specified trace events in order
|
||||
to facilitate early boot debugging.
|
||||
See also Documentation/trace/events.txt
|
||||
to facilitate early boot debugging. The event-list is a
|
||||
comma separated list of trace events to enable. See
|
||||
also Documentation/trace/events.txt
|
||||
|
||||
trace_options=[option-list]
|
||||
[FTRACE] Enable or disable tracer options at boot.
|
||||
|
|
|
|||
|
|
@ -602,7 +602,7 @@ These currently include
|
|||
|
||||
stripe_cache_size (currently raid5 only)
|
||||
number of entries in the stripe cache. This is writable, but
|
||||
there are upper and lower limits (32768, 16). Default is 128.
|
||||
there are upper and lower limits (32768, 17). Default is 256.
|
||||
strip_cache_active (currently raid5 only)
|
||||
number of active entries in the stripe cache
|
||||
preread_bypass_threshold (currently raid5 only)
|
||||
|
|
|
|||
|
|
@ -1538,9 +1538,9 @@ set_cmdline(struct mic_info *mic)
|
|||
|
||||
len = snprintf(buffer, PATH_MAX,
|
||||
"clocksource=tsc highres=off nohz=off ");
|
||||
len += snprintf(buffer + len, PATH_MAX,
|
||||
len += snprintf(buffer + len, PATH_MAX - len,
|
||||
"cpufreq_on;corec6_off;pc3_off;pc6_off ");
|
||||
len += snprintf(buffer + len, PATH_MAX,
|
||||
len += snprintf(buffer + len, PATH_MAX - len,
|
||||
"ifcfg=static;address,172.31.%d.1;netmask,255.255.255.0",
|
||||
mic->id + 1);
|
||||
|
||||
|
|
|
|||
|
|
@ -183,8 +183,9 @@ provide meaningful defenses.
|
|||
### Canaries, blinding, and other secrets
|
||||
|
||||
It should be noted that things like the stack canary discussed earlier
|
||||
are technically statistical defenses, since they rely on a (leakable)
|
||||
secret value.
|
||||
are technically statistical defenses, since they rely on a secret value,
|
||||
and such values may become discoverable through an information exposure
|
||||
flaw.
|
||||
|
||||
Blinding literal values for things like JITs, where the executable
|
||||
contents may be partially under the control of userspace, need a similar
|
||||
|
|
@ -199,8 +200,8 @@ working?) in order to maximize their success.
|
|||
Since the location of kernel memory is almost always instrumental in
|
||||
mounting a successful attack, making the location non-deterministic
|
||||
raises the difficulty of an exploit. (Note that this in turn makes
|
||||
the value of leaks higher, since they may be used to discover desired
|
||||
memory locations.)
|
||||
the value of information exposures higher, since they may be used to
|
||||
discover desired memory locations.)
|
||||
|
||||
#### Text and module base
|
||||
|
||||
|
|
@ -222,14 +223,21 @@ become more difficult to locate.
|
|||
Much of the kernel's dynamic memory (e.g. kmalloc, vmalloc, etc) ends up
|
||||
being relatively deterministic in layout due to the order of early-boot
|
||||
initializations. If the base address of these areas is not the same
|
||||
between boots, targeting them is frustrated, requiring a leak specific
|
||||
to the region.
|
||||
between boots, targeting them is frustrated, requiring an information
|
||||
exposure specific to the region.
|
||||
|
||||
#### Structure layout
|
||||
|
||||
By performing a per-build randomization of the layout of sensitive
|
||||
structures, attacks must either be tuned to known kernel builds or expose
|
||||
enough kernel memory to determine structure layouts before manipulating
|
||||
them.
|
||||
|
||||
|
||||
## Preventing Leaks
|
||||
## Preventing Information Exposures
|
||||
|
||||
Since the locations of sensitive structures are the primary target for
|
||||
attacks, it is important to defend against leaks of both kernel memory
|
||||
attacks, it is important to defend against exposure of both kernel memory
|
||||
addresses and kernel memory contents (since they may contain kernel
|
||||
addresses or other sensitive things like canary values).
|
||||
|
||||
|
|
@ -250,8 +258,8 @@ sure structure holes are cleared.
|
|||
When releasing memory, it is best to poison the contents (clear stack on
|
||||
syscall return, wipe heap memory on a free), to avoid reuse attacks that
|
||||
rely on the old contents of memory. This frustrates many uninitialized
|
||||
variable attacks, stack info leaks, heap info leaks, and use-after-free
|
||||
attacks.
|
||||
variable attacks, stack content exposures, heap content exposures, and
|
||||
use-after-free attacks.
|
||||
|
||||
### Destination tracking
|
||||
|
||||
|
|
|
|||
18
Documentation/sphinx/convert_template.sed
Normal file
18
Documentation/sphinx/convert_template.sed
Normal file
|
|
@ -0,0 +1,18 @@
|
|||
#
|
||||
# Pandoc doesn't grok <function> or <structname>, so convert them
|
||||
# ahead of time.
|
||||
#
|
||||
# Use the following escapes to pass through pandoc:
|
||||
# $bq = "`"
|
||||
# $lt = "<"
|
||||
# $gt = ">"
|
||||
#
|
||||
s%<function>\([^<(]\+\)()</function>%:c:func:$bq\1()$bq%g
|
||||
s%<function>\([^<(]\+\)</function>%:c:func:$bq\1()$bq%g
|
||||
s%<structname>struct *\([^<]\+\)</structname>%:c:type:$bqstruct \1 $lt\1$gt$bq%g
|
||||
s%struct <structname>\([^<]\+\)</structname>%:c:type:$bqstruct \1 $lt\1$gt$bq%g
|
||||
s%<structname>\([^<]\+\)</structname>%:c:type:$bqstruct \1 $lt\1$gt$bq%g
|
||||
#
|
||||
# Wrap docproc directives in para and code blocks.
|
||||
#
|
||||
s%^\(!.*\)$%<para><code>DOCPROC: \1</code></para>%
|
||||
141
Documentation/sphinx/kernel-doc.py
Normal file
141
Documentation/sphinx/kernel-doc.py
Normal file
|
|
@ -0,0 +1,141 @@
|
|||
# coding=utf-8
|
||||
#
|
||||
# Copyright © 2016 Intel Corporation
|
||||
#
|
||||
# Permission is hereby granted, free of charge, to any person obtaining a
|
||||
# copy of this software and associated documentation files (the "Software"),
|
||||
# to deal in the Software without restriction, including without limitation
|
||||
# the rights to use, copy, modify, merge, publish, distribute, sublicense,
|
||||
# and/or sell copies of the Software, and to permit persons to whom the
|
||||
# Software is furnished to do so, subject to the following conditions:
|
||||
#
|
||||
# The above copyright notice and this permission notice (including the next
|
||||
# paragraph) shall be included in all copies or substantial portions of the
|
||||
# Software.
|
||||
#
|
||||
# THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
|
||||
# IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
|
||||
# FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL
|
||||
# THE AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
|
||||
# LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING
|
||||
# FROM, OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS
|
||||
# IN THE SOFTWARE.
|
||||
#
|
||||
# Authors:
|
||||
# Jani Nikula <jani.nikula@intel.com>
|
||||
#
|
||||
# Please make sure this works on both python2 and python3.
|
||||
#
|
||||
|
||||
import os
|
||||
import subprocess
|
||||
import sys
|
||||
import re
|
||||
import glob
|
||||
|
||||
from docutils import nodes, statemachine
|
||||
from docutils.statemachine import ViewList
|
||||
from docutils.parsers.rst import directives
|
||||
from sphinx.util.compat import Directive
|
||||
from sphinx.ext.autodoc import AutodocReporter
|
||||
|
||||
class KernelDocDirective(Directive):
|
||||
"""Extract kernel-doc comments from the specified file"""
|
||||
required_argument = 1
|
||||
optional_arguments = 4
|
||||
option_spec = {
|
||||
'doc': directives.unchanged_required,
|
||||
'functions': directives.unchanged_required,
|
||||
'export': directives.unchanged,
|
||||
'internal': directives.unchanged,
|
||||
}
|
||||
has_content = False
|
||||
|
||||
def run(self):
|
||||
env = self.state.document.settings.env
|
||||
cmd = [env.config.kerneldoc_bin, '-rst', '-enable-lineno']
|
||||
|
||||
filename = env.config.kerneldoc_srctree + '/' + self.arguments[0]
|
||||
export_file_patterns = []
|
||||
|
||||
# Tell sphinx of the dependency
|
||||
env.note_dependency(os.path.abspath(filename))
|
||||
|
||||
tab_width = self.options.get('tab-width', self.state.document.settings.tab_width)
|
||||
|
||||
# FIXME: make this nicer and more robust against errors
|
||||
if 'export' in self.options:
|
||||
cmd += ['-export']
|
||||
export_file_patterns = str(self.options.get('export')).split()
|
||||
elif 'internal' in self.options:
|
||||
cmd += ['-internal']
|
||||
export_file_patterns = str(self.options.get('internal')).split()
|
||||
elif 'doc' in self.options:
|
||||
cmd += ['-function', str(self.options.get('doc'))]
|
||||
elif 'functions' in self.options:
|
||||
for f in str(self.options.get('functions')).split():
|
||||
cmd += ['-function', f]
|
||||
|
||||
for pattern in export_file_patterns:
|
||||
for f in glob.glob(env.config.kerneldoc_srctree + '/' + pattern):
|
||||
env.note_dependency(os.path.abspath(f))
|
||||
cmd += ['-export-file', f]
|
||||
|
||||
cmd += [filename]
|
||||
|
||||
try:
|
||||
env.app.verbose('calling kernel-doc \'%s\'' % (" ".join(cmd)))
|
||||
|
||||
p = subprocess.Popen(cmd, stdout=subprocess.PIPE, stderr=subprocess.PIPE, universal_newlines=True)
|
||||
out, err = p.communicate()
|
||||
|
||||
# python2 needs conversion to unicode.
|
||||
# python3 with universal_newlines=True returns strings.
|
||||
if sys.version_info.major < 3:
|
||||
out, err = unicode(out, 'utf-8'), unicode(err, 'utf-8')
|
||||
|
||||
if p.returncode != 0:
|
||||
sys.stderr.write(err)
|
||||
|
||||
env.app.warn('kernel-doc \'%s\' failed with return code %d' % (" ".join(cmd), p.returncode))
|
||||
return [nodes.error(None, nodes.paragraph(text = "kernel-doc missing"))]
|
||||
elif env.config.kerneldoc_verbosity > 0:
|
||||
sys.stderr.write(err)
|
||||
|
||||
lines = statemachine.string2lines(out, tab_width, convert_whitespace=True)
|
||||
result = ViewList()
|
||||
|
||||
lineoffset = 0;
|
||||
line_regex = re.compile("^#define LINENO ([0-9]+)$")
|
||||
for line in lines:
|
||||
match = line_regex.search(line)
|
||||
if match:
|
||||
# sphinx counts lines from 0
|
||||
lineoffset = int(match.group(1)) - 1
|
||||
# we must eat our comments since the upset the markup
|
||||
else:
|
||||
result.append(line, filename, lineoffset)
|
||||
lineoffset += 1
|
||||
|
||||
node = nodes.section()
|
||||
buf = self.state.memo.title_styles, self.state.memo.section_level, self.state.memo.reporter
|
||||
self.state.memo.reporter = AutodocReporter(result, self.state.memo.reporter)
|
||||
self.state.memo.title_styles, self.state.memo.section_level = [], 0
|
||||
try:
|
||||
self.state.nested_parse(result, 0, node, match_titles=1)
|
||||
finally:
|
||||
self.state.memo.title_styles, self.state.memo.section_level, self.state.memo.reporter = buf
|
||||
|
||||
return node.children
|
||||
|
||||
except Exception as e: # pylint: disable=W0703
|
||||
env.app.warn('kernel-doc \'%s\' processing failed with: %s' %
|
||||
(" ".join(cmd), str(e)))
|
||||
return [nodes.error(None, nodes.paragraph(text = "kernel-doc missing"))]
|
||||
|
||||
def setup(app):
|
||||
app.add_config_value('kerneldoc_bin', None, 'env')
|
||||
app.add_config_value('kerneldoc_srctree', None, 'env')
|
||||
app.add_config_value('kerneldoc_verbosity', 1, 'env')
|
||||
|
||||
app.add_directive('kernel-doc', KernelDocDirective)
|
||||
23
Documentation/sphinx/post_convert.sed
Normal file
23
Documentation/sphinx/post_convert.sed
Normal file
|
|
@ -0,0 +1,23 @@
|
|||
#
|
||||
# Unescape.
|
||||
#
|
||||
s/$bq/`/g
|
||||
s/$lt/</g
|
||||
s/$gt/>/g
|
||||
#
|
||||
# pandoc thinks that both "_" needs to be escaped. Remove the extra
|
||||
# backslashes.
|
||||
#
|
||||
s/\\_/_/g
|
||||
#
|
||||
# Unwrap docproc directives.
|
||||
#
|
||||
s/^``DOCPROC: !E\(.*\)``$/.. kernel-doc:: \1\n :export:/
|
||||
s/^``DOCPROC: !I\(.*\)``$/.. kernel-doc:: \1\n :internal:/
|
||||
s/^``DOCPROC: !F\([^ ]*\) \(.*\)``$/.. kernel-doc:: \1\n :functions: \2/
|
||||
s/^``DOCPROC: !P\([^ ]*\) \(.*\)``$/.. kernel-doc:: \1\n :doc: \2/
|
||||
s/^``DOCPROC: \(!.*\)``$/.. WARNING: DOCPROC directive not supported: \1/
|
||||
#
|
||||
# Trim trailing whitespace.
|
||||
#
|
||||
s/[[:space:]]*$//
|
||||
Some files were not shown because too many files have changed in this diff Show more
Loading…
Reference in a new issue