Pull f2fs update from Jaegeuk Kim:
"In this round, we've mainly focused on performance tuning and critical
bug fixes occurred in low-end devices. Sheng Yong introduced
lost_found feature to keep missing files during recovery instead of
thrashing them. We're preparing coming fsverity implementation. And,
we've got more features to communicate with users for better
performance. In low-end devices, some memory-related issues were
fixed, and subtle race condtions and corner cases were addressed as
well.
Enhancements:
- large nat bitmaps for more free node ids
- add three block allocation policies to pass down write hints given by user
- expose extension list to user and introduce hot file extension
- tune small devices seamlessly for low-end devices
- set readdir_ra by default
- give more resources under gc_urgent mode regarding to discard and cleaning
- introduce fsync_mode to enforce posix or not
- nowait aio support
- add lost_found feature to keep dangling inodes
- reserve bits for future fsverity feature
- add test_dummy_encryption for FBE
Bug fixes:
- don't use highmem for dentry pages
- align memory boundary for bitops
- truncate preallocated blocks in write errors
- guarantee i_times on fsync call
- clear CP_TRIMMED_FLAG correctly
- prevent node chain loop during recovery
- avoid data race between atomic write and background cleaning
- avoid unnecessary selinux violation warnings on resgid option
- GFP_NOFS to avoid deadlock in quota and read paths
- fix f2fs_skip_inode_update to allow i_size recovery
In addition to the above, there are several minor bug fixes and clean-ups"
Cherry-pick from origin/upstream-f2fs-stable-linux-4.9.y:
ac389af190 f2fs: remain written times to update inode during fsync
270deeb871 f2fs: make assignment of t->dentry_bitmap more readable
a4fa11c8da f2fs: truncate preallocated blocks in error case
4478970f0e f2fs: fix a wrong condition in f2fs_skip_inode_update
29cead58f5 f2fs: reserve bits for fs-verity
848b293a5d f2fs: Add a segment type check in inplace write
2dc8f5a3a6 f2fs: no need to initialize zero value for GFP_F2FS_ZERO
83b9bb95a6 f2fs: don't track new nat entry in nat set
a33ce03ac4 f2fs: clean up with F2FS_BLK_ALIGN
a3f8ec8082 f2fs: check blkaddr more accuratly before issue a bio
034f11eadb f2fs: Set GF_NOFS in read_cache_page_gfp while doing f2fs_quota_read
aa5bcfd8f4 f2fs: introduce a new mount option test_dummy_encryption
9b880fe6e6 f2fs: introduce F2FS_FEATURE_LOST_FOUND feature
80d6489a08 f2fs: release locks before return in f2fs_ioc_gc_range()
9f1896c490 f2fs: align memory boundary for bitops
c7930ee883 f2fs: remove unneeded set_cold_node()
355d234640 f2fs: add nowait aio support
e9a50e6b94 f2fs: wrap all options with f2fs_sb_info.mount_opt
b6d2ec83e0 f2fs: Don't overwrite all types of node to keep node chain
9a95481629 f2fs: introduce mount option for fsync mode
4ce4eb6970 f2fs: fix to restore old mount option in ->remount_fs
8f711c344e f2fs: wrap sb_rdonly with f2fs_readonly
c07478ee84 f2fs: avoid selinux denial on CAP_SYS_RESOURCE
ac734c416f f2fs: support hot file extension
f4f10221ac f2fs: fix to avoid race in between atomic write and background GC
e87b13ec16 f2fs: do gc in greedy mode for whole range if gc_urgent mode is set
e9878588de f2fs: issue discard aggressively in the gc_urgent mode
ad3ce479e6 f2fs: set readdir_ra by default
5aae2026bb f2fs: add auto tuning for small devices
78c1fc2d8f f2fs: add mount option for segment allocation policy
ecd02f5646 f2fs: don't stop GC if GC is contended
1e72cb27d2 f2fs: expose extension_list sysfs entry
061839d178 f2fs: fix to set KEEP_SIZE bit in f2fs_zero_range
4951ebcbc4 f2fs: introduce sb_lock to make encrypt pwsalt update exclusive
939f6be042 f2fs: remove redundant initialization of pointer 'p'
39bea4bc8e f2fs: flush cp pack except cp pack 2 page at first
770611eb2a f2fs: clean up f2fs_sb_has_xxx functions
4d8e4a8965 f2fs: remove redundant check of page type when submit bio
e9878588de f2fs: issue discard aggressively in the gc_urgent mode
ad3ce479e6 f2fs: set readdir_ra by default
5aae2026bb f2fs: add auto tuning for small devices
78c1fc2d8f f2fs: add mount option for segment allocation policy
ecd02f5646 f2fs: don't stop GC if GC is contended
1e72cb27d2 f2fs: expose extension_list sysfs entry
061839d178 f2fs: fix to set KEEP_SIZE bit in f2fs_zero_range
4951ebcbc4 f2fs: introduce sb_lock to make encrypt pwsalt update exclusive
939f6be042 f2fs: remove redundant initialization of pointer 'p'
39bea4bc8e f2fs: flush cp pack except cp pack 2 page at first
770611eb2a f2fs: clean up f2fs_sb_has_xxx functions
4d8e4a8965 f2fs: remove redundant check of page type when submit bio
b57a37f01f f2fs: fix to handle looped node chain during recovery
9ac5b8c540 f2fs: handle quota for orphan inodes
87c1806601 f2fs: support passing down write hints to block layer with F2FS policy
bcdc571e8d f2fs: support passing down write hints given by users to block layer
92413bc12e f2fs: fix to clear CP_TRIMMED_FLAG
a1afb55f97 f2fs: support large nat bitmap
6360391404 f2fs: fix to check extent cache in f2fs_drop_extent_tree
7de4fccdbc f2fs: restrict inline_xattr_size configuration
aae506a8b7 f2fs: fix heap mode to reset it back
8fa455bb6e f2fs: fix potential corruption in area before F2FS_SUPER_OFFSET
9d9cb0ef73 fscrypt: fix build with pre-4.6 gcc versions
401052ffc6 fscrypt: remove 'ci' parameter from fscrypt_put_encryption_info()
549b2061b3 fscrypt: fix up fscrypt_fname_encrypted_size() for internal use
c440b5091a fscrypt: define fscrypt_fname_alloc_buffer() to be for presented names
7d82f0e1c3 ext4: switch to fscrypt ->symlink() helper functions
ba4efe5604 ext4: switch to fscrypt_get_symlink()
b0edc2f22d fscrypt: calculate NUL-padding length in one place only
62cfdd9868 fscrypt: move fscrypt_symlink_data to fscrypt_private.h
e4e6776522 fscrypt: remove fscrypt_fname_usr_to_disk()
45028b5aaa f2fs: switch to fscrypt_get_symlink()
f62d3d31e0 f2fs: switch to fscrypt ->symlink() helper functions
da32a1633a fscrypt: new helper function - fscrypt_get_symlink()
a7e05c731d fscrypt: new helper functions for ->symlink()
eb9c5fd896 fscrypt: trim down fscrypt.h includes
0a02472d8a fscrypt: move fscrypt_is_dot_dotdot() to fs/crypto/fname.c
9d51ca8027 fscrypt: move fscrypt_valid_enc_modes() to fscrypt_private.h
efbfa8c6a0 fscrypt: move fscrypt_operations declaration to fscrypt_supp.h
616dbd2bdc fscrypt: split fscrypt_dummy_context_enabled() into supp/notsupp versions
f0c472bcbf fscrypt: move fscrypt_ctx declaration to fscrypt_supp.h
bc76f39109 fscrypt: move fscrypt_info_cachep declaration to fscrypt_private.h
b67b07ec49 fscrypt: move fscrypt_control_page() to supp/notsupp headers
d8dfb89961 fscrypt: move fscrypt_has_encryption_key() to supp/notsupp headers
Signed-off-by: Jaegeuk Kim <jaegeuk@google.com>
Cherry-picked from upstream-f2fs-stable-linux-4.9.y
Changes include:
commit 30da3a4de9 ("f2fs: hurry up to issue discard after io interruption")
commit d1c363b483 ("f2fs: fix to show correct discard_granularity in sysfs")
...
commit e6b120d4d0 ("f2fs/fscrypt: catch up to v4.12")
commit 4d7931d727 ("KEYS: Differentiate uses of rcu_dereference_key() and user_key_payload()")
Signed-off-by: Hyojun Kim <hyojun@google.com>
Hyojun Kimauthored and Greg Kroah-Hartmancommited 2017-10-13 11:27:08 +00:00
Make inline_dentry as default mount option to improve space usage and
IO performance in scenario of numerous small directory.
It adds noinline_dentry mount option, instead.
Signed-off-by: Chao Yu <yuchao0@huawei.com>
Signed-off-by: Jaegeuk Kim <jaegeuk@kernel.org>
Chao Yuauthored and Jaegeuk Kimcommited 2016-08-29 18:31:17 -07:00
This mount option is to enable original log-structured filesystem forcefully.
So, there should be no random writes for main area.
Especially, this supports host-managed SMR device.
Signed-off-by: Jaegeuk Kim <jaegeuk@kernel.org>
Add a new option 'data_flush' to enable data flush functionality.
Signed-off-by: Chao Yu <chao2.yu@samsung.com>
Signed-off-by: Jaegeuk Kim <jaegeuk@kernel.org>
Chao Yuauthored and Jaegeuk Kimcommited 2015-12-16 09:25:48 -08:00
This patch fix some typos in Documentation/filesystems/f2fs.txt
Signed-off-by: Masanari Iida <standby24x7@gmail.com>
Acked-by: Randy Dunlap <rdunlap@infradead.org>
Signed-off-by: Jaegeuk Kim <jaegeuk@kernel.org>
Masanari Iidaauthored and Jaegeuk Kimcommited 2015-12-04 11:52:33 -08:00
Enable inline_data feature by default since it brings us better
performance and space utilization and now has already stable.
Add another option noinline_data to disable it during mount.
Suggested-by: Jaegeuk Kim <jaegeuk@kernel.org>
Suggested-by: Chao Yu <chao2.yu@samsung.com>
Signed-off-by: Wanpeng Li <wanpeng.li@linux.intel.com>
Signed-off-by: Jaegeuk Kim <jaegeuk@kernel.org>
Wanpeng Liauthored and Jaegeuk Kimcommited 2015-04-10 15:08:52 -07:00
This patch adds a mount option 'extent_cache' in f2fs.
It is try to use a rb-tree based extent cache to cache more mapping information
with less memory if this option is set, otherwise we will use the original one
extent info cache.
Suggested-by: Changman Lee <cm224.lee@samsung.com>
Signed-off-by: Chao Yu <chao2.yu@samsung.com>
Signed-off-by: Jaegeuk Kim <jaegeuk@kernel.org>
Chao Yuauthored and Jaegeuk Kimcommited 2015-03-03 09:58:46 -08:00
This patch introduces a batched trimming feature, which submits split discard
commands.
This is to avoid long latency due to huge trim commands.
If fstrim was triggered ranging from 0 to the end of device, we should lock
all the checkpoint-related mutexes, resulting in very long latency.
Reviewed-by: Chao Yu <chao2.yu@samsung.com>
Signed-off-by: Jaegeuk Kim <jaegeuk@kernel.org>
This patch adds a mount option, norecovery, which is mostly same as
disable_roll_forward. The only difference is that norecovery should be activated
with read-only mount option.
This can be used when user wants to check whether f2fs is mountable or not
without any recovery process. (e.g., xfstests/200)
Reviewed-by: Chao Yu <chao2.yu@samsung.com>
Signed-off-by: Jaegeuk Kim <jaegeuk@kernel.org>
If a system wants to reduce the booting time as a top priority, now we can
use a mount option, -o fastboot.
With this option, f2fs conducts a little bit slow write_checkpoint, but
it can avoid the node page reads during the next mount time.
Signed-off-by: Jaegeuk Kim <jaegeuk@kernel.org>
This patch adds descriptions for the inline dir support in f2fs document.
Signed-off-by: Chao Yu <chao2.yu@samsung.com>
Signed-off-by: Jaegeuk Kim <jaegeuk@kernel.org>
Chao Yuauthored and Jaegeuk Kimcommited 2014-11-03 16:07:32 -08:00
This patch changes the ipu_policy setting to use any combination of orthogonal policies.
Signed-off-by: Changman Lee <cm224.lee@samsung.com>
Signed-off-by: Jaegeuk Kim <jaegeuk@kernel.org>
If user wrote F2FS_IPU_FSYNC:4 in /sys/fs/f2fs/ipu_policy, f2fs_sync_file
only starts to try in-place-updates.
And, if the number of dirty pages is over /sys/fs/f2fs/min_fsync_blocks, it
keeps out-of-order manner. Otherwise, it triggers in-place-updates.
This may be used by storage showing very high random write performance.
For example, it can be used when,
Seq. writes (Data) + wait + Seq. writes (Node)
is pretty much slower than,
Rand. writes (Data)
Signed-off-by: Jaegeuk Kim <jaegeuk@kernel.org>
This patch adds a mount option, nobarrier, in f2fs.
The assumption in here is that file system keeps the IO ordering, but
doesn't care about cache flushes inside the storages.
Reviewed-by: Chao Yu <chao2.yu@samsung.com>
Signed-off-by: Jaegeuk Kim <jaegeuk@kernel.org>
When large directory feathure is enable, We have one case which could cause
overflow in dir_buckets() as following:
special case: level + dir_level >= 32 and level < MAX_DIR_HASH_DEPTH / 2.
Here we define MAX_DIR_BUCKETS to limit the return value when the condition
could trigger potential overflow.
Changes from V1
o modify description of calculation in f2fs.txt suggested by Changman Lee.
Suggested-by: Changman Lee <cm224.lee@samsung.com>
Signed-off-by: Chao Yu <chao2.yu@samsung.com>
Signed-off-by: Jaegeuk Kim <jaegeuk@kernel.org>
Chao Yuauthored and Jaegeuk Kimcommited 2014-06-04 13:34:30 +09:00
Some storage devices show relatively high latencies to complete cache_flush
commands, even though their normal IO speed is prettry much high. In such
the case, it needs to merge cache_flush commands as much as possible to avoid
issuing them redundantly.
So, this patch introduces a mount option, "-o flush_merge", to mitigate such
the overhead.
If this option is enabled by user, F2FS merges the cache_flush commands and then
issues just one cache_flush on behalf of them. Once the single command is
finished, F2FS sends a completion signal to all the pending threads.
Note that, this option can be used under a workload consisting of very intensive
concurrent fsync calls, while the storage handles cache_flush commands slowly.
Signed-off-by: Jaegeuk Kim <jaegeuk.kim@samsung.com>
It is more reasonable to determine the reclaiming rate of prefree segments
according to the volume size, which is set to 5% by default.
For example, if the volume is 128GB, the prefree segments are reclaimed
when the number reaches to 6.4GB.
Signed-off-by: Jaegeuk Kim <jaegeuk.kim@samsung.com>
This patch introduces ram_thresh, a sysfs entry, which controls the memory
footprint used by the free nid list and the nat cache.
Previously, the free nid list was controlled by MAX_FREE_NIDS, while the nat
cache was managed by NM_WOUT_THRESHOLD.
However, this approach cannot be applied dynamically according to the system.
So, this patch adds ram_thresh that users can specify the threshold, which is
in order of 1 / 1024.
For example, if the total ram size is 4GB and the value is set to 10 by default,
f2fs tries to control the number of free nids and nat caches not to consume over
10 * (4GB / 1024) = 10MB.
Signed-off-by: Jaegeuk Kim <jaegeuk.kim@samsung.com>
This patch adds an sysfs entry to control dir_level used by the large directory.
The description of this entry is:
dir_level This parameter controls the directory level to
support large directory. If a directory has a
number of files, it can reduce the file lookup
latency by increasing this dir_level value.
Otherwise, it needs to decrease this value to
reduce the space overhead. The default value is 0.
Signed-off-by: Jaegeuk Kim <jaegeuk.kim@samsung.com>
This patch introduces an i_dir_level field to support large directory.
Previously, f2fs maintains multi-level hash tables to find a dentry quickly
from a bunch of chiild dentries in a directory, and the hash tables consist of
the following tree structure as below.
In Documentation/filesystems/f2fs.txt,
----------------------
A : bucket
B : block
N : MAX_DIR_HASH_DEPTH
----------------------
level #0 | A(2B)
|
level #1 | A(2B) - A(2B)
|
level #2 | A(2B) - A(2B) - A(2B) - A(2B)
. | . . . .
level #N/2 | A(2B) - A(2B) - A(2B) - A(2B) - A(2B) - ... - A(2B)
. | . . . .
level #N | A(4B) - A(4B) - A(4B) - A(4B) - A(4B) - ... - A(4B)
But, if we can guess that a directory will handle a number of child files,
we don't need to traverse the tree from level #0 to #N all the time.
Since the lower level tables contain relatively small number of dentries,
the miss ratio of the target dentry is likely to be high.
In order to avoid that, we can configure the hash tables sparsely from level #0
like this.
level #0 | A(2B) - A(2B) - A(2B) - A(2B)
level #1 | A(2B) - A(2B) - A(2B) - A(2B) - A(2B) - ... - A(2B)
. | . . . .
level #N/2 | A(2B) - A(2B) - A(2B) - A(2B) - A(2B) - ... - A(2B)
. | . . . .
level #N | A(4B) - A(4B) - A(4B) - A(4B) - A(4B) - ... - A(4B)
With this structure, we can skip the ineffective tree searches in lower level
hash tables.
This patch adds just a facility for this by introducing i_dir_level in
f2fs_inode.
Signed-off-by: Jaegeuk Kim <jaegeuk.kim@samsung.com>
This patch adds missing some description of sysfs entries in
- Documentation/ABI/testing/sysfs-fs-f2fs
- Documentation/filesystems/f2fs.txt.
And it adds a maintained document entry of F2FS in MAINTAINERS.
Signed-off-by: Jaegeuk Kim <jaegeuk.kim@samsung.com>
This patch describes the inline_data support in f2fs document.
Signed-off-by: Huajun Li <huajun.li@intel.com>
Signed-off-by: Jaegeuk Kim <jaegeuk.kim@samsung.com>
Huajun Liauthored and Jaegeuk Kimcommited 2013-12-26 20:40:44 +09:00
This patch introduces new sysfs entries for users to control the policy of
in-place-updates, namely IPU, in f2fs.
Sometimes f2fs suffers from performance degradation due to its out-of-place
update policy that produces many additional node block writes.
If the storage performance is very dependant on the amount of data writes
instead of IO patterns, we'd better drop this out-of-place update policy.
This patch suggests 5 polcies and their triggering conditions as follows.
[sysfs entry name = ipu_policy]
0: F2FS_IPU_FORCE all the time,
1: F2FS_IPU_SSR if SSR mode is activated,
2: F2FS_IPU_UTIL if FS utilization is over threashold,
3: F2FS_IPU_SSR_UTIL if SSR mode is activated and FS utilization is over
threashold,
4: F2FS_IPU_DISABLE disable IPU. (=default option)
[sysfs entry name = min_ipu_util]
This parameter controls the threshold to trigger in-place-updates.
The number indicates percentage of the filesystem utilization, and used by
F2FS_IPU_UTIL and F2FS_IPU_SSR_UTIL policies.
For more details, see need_inplace_update() in segment.h.
Signed-off-by: Jaegeuk Kim <jaegeuk.kim@samsung.com>
This patch adds a control method in sysfs to reclaim prefree segments.
Signed-off-by: Changman Lee <cm224.lee@samsung.com>
Signed-off-by: Jaegeuk Kim <jaegeuk.kim@samsung.com>
Add sysfs entry gc_idle to control the gc policy. Where
gc_idle = 1 corresponds to selecting a cost benefit approach,
while gc_idle = 2 corresponds to selecting a greedy approach
to garbage collection. The selection is mutually exclusive one
approach will work at any point. If gc_idle = 0, then this
option is disabled.
Cc: Gu Zheng <guz.fnst@cn.fujitsu.com>
Signed-off-by: Namjae Jeon <namjae.jeon@samsung.com>
Signed-off-by: Pankaj Kumar <pankaj.km@samsung.com>
Reviewed-by: Gu Zheng <guz.fnst@cn.fujitsu.com>
[Jaegeuk Kim: change the select_gc_type() flow slightly]
Signed-off-by: Jaegeuk Kim <jaegeuk.kim@samsung.com>
Namjae Jeonauthored and Jaegeuk Kimcommited 2013-08-06 22:00:18 +09:00
Add sysfs entries to control the timing parameters for
f2fs gc thread.
Various Sysfs options introduced are:
gc_min_sleep_time: Min Sleep time for GC in ms
gc_max_sleep_time: Max Sleep time for GC in ms
gc_no_gc_sleep_time: Default Sleep time for GC in ms
Cc: Gu Zheng <guz.fnst@cn.fujitsu.com>
Signed-off-by: Namjae Jeon <namjae.jeon@samsung.com>
Signed-off-by: Pankaj Kumar <pankaj.km@samsung.com>
Reviewed-by: Gu Zheng <guz.fnst@cn.fujitsu.com>
[Jaegeuk Kim: fix an umount bug and some minor changes]
Signed-off-by: Jaegeuk Kim <jaegeuk.kim@samsung.com>
Namjae Jeonauthored and Jaegeuk Kimcommited 2013-08-06 21:53:34 +09:00
This patch adds some description on fsck.f2fs and dump.f2fs which is
recently merged into f2fs-tools.
Signed-off-by: Changman Lee <cm224.lee@samsung.com>
Signed-off-by: Jaegeuk Kim <jaegeuk.kim@samsung.com>
Changman Leeauthored and Jaegeuk Kimcommited 2013-07-30 15:17:02 +09:00
Add the f2fs_remount function call which will be used
during the filesystem remounting. This function
will help us to change the mount options specific to
f2fs.
Also modify the f2fs background_gc mount option, which
will allow the user to dynamically trun on/off the
garbage collection in f2fs based on the background_gc
value. If background_gc=on, Garbage collection will
be turned off & if background_gc=off, Garbage collection
will be truned on.
By default the garbage collection is on in f2fs.
Change Log:
v2: Incorporated the review comments by Gu Zheng.
Removing the restore part for VFS flags
Updating comments with proper flag conditions
Display GC background option as ON/OFF
Revised conditions to stop GC in case of remount
v1: Initial changes for adding remount_fs callback
support.
Cc: Gu Zheng <guz.fnst@cn.fujitsu.com>
Signed-off-by: Namjae Jeon <namjae.jeon@samsung.com>
Signed-off-by: Pankaj Kumar <pankaj.km@samsung.com>
Reviewed-by: Gu Zheng <guz.fnst@cn.fujitsu.com>
[Jaegeuk Kim: change /** with /* for the coding style]
Signed-off-by: Jaegeuk Kim <jaegeuk.kim@samsung.com>
Namjae Jeonauthored and Jaegeuk Kimcommited 2013-06-17 21:17:55 +09:00
o mkfs.f2fs supports no discard option.
o fixed volume label size in 512 bytes.
Signed-off-by: Changman Lee <cm224.lee@samsung.com>
Signed-off-by: Jaegeuk Kim <jaegeuk.kim@samsung.com>
Changman Leeauthored and Jaegeuk Kimcommited 2013-04-03 17:27:52 +09:00
document to reflect the layout generated by mkfs.f2fs .
Signed-off-by: Huajun Li <huajun.li.lee@gmail.com>
Signed-off-by: Jaegeuk Kim <jaegeuk.kim@samsung.com>
Huajun Liauthored and Jaegeuk Kimcommited 2013-01-04 09:42:59 +09:00
In f2fs_fs.h, one f2fs inode contains 923 data block pointers, while
f2fs documentation says it is 929. Fix this inconsistence.
Signed-off-by: Huajun Li <huajun.li.lee@gmail.com>
Huajun Liauthored and Jaegeuk Kimcommited 2012-12-11 13:43:44 +09:00
I moved the f2fs-tools.git into kernel.org.
And I added a new mailing list, linux-f2fs-devel@lists.sourceforge.net.
Signed-off-by: Jaegeuk Kim <jaegeuk.kim@samsung.com>
This adds a document describing the mount options, proc entries, usage, and
design of Flash-Friendly File System, namely F2FS.
Signed-off-by: Jaegeuk Kim <jaegeuk.kim@samsung.com>