Initial commit

This commit is contained in:
2025-03-07 19:22:02 +01:00
commit 4a98255d83
55743 changed files with 5280367 additions and 0 deletions
+68
View File
@@ -0,0 +1,68 @@
3.0.0 / 2018-08-01
------------------
- **BREAKING:** Follow symlinks by default (use the new `preserveSymlinks` option to get the old behavior) [#29](https://github.com/jprichardson/node-klaw/pull/29)
- **BREAKING:** Drop Node v4 support
2.1.1 / 2017-11-18
------------------
- Performance optimization [#27](https://github.com/jprichardson/node-klaw/pull/27)
2.1.0 / 2017-08-10
------------------
### Added
- Added `depthLimit` option to limit how deep to recurse into folders. [#25](https://github.com/jprichardson/node-klaw/pull/25)
2.0.0 / 2017-06-23
------------------
### Changed
- `graceful-fs` is now a regular dependency, and is always loaded. This should speed up `require` time
- Dropped support for Node 0.10 & 0.12 and io.js
1.3.1 / 2016-10-25
------------------
### Added
- `graceful-fs` added as an `optionalDependencies`. Thanks [ryanzim]!
1.3.0 / 2016-06-09
------------------
### Added
- `filter` option to pre-filter and not walk directories.
1.2.0 / 2016-04-16
------------------
- added support for custom `fs` implementation. Useful for https://github.com/tschaub/mock-fs
1.1.3 / 2015-12-23
------------------
- bugfix: if `readdir` error, got hung up. See: https://github.com/jprichardson/node-klaw/issues/1
1.1.2 / 2015-11-12
------------------
- assert that param `dir` is a `string`
1.1.1 / 2015-10-25
------------------
- bug fix, options not being passed
1.1.0 / 2015-10-25
------------------
- added `queueMethod` and `pathSorter` to `options` to affect searching strategy.
1.0.0 / 2015-10-25
------------------
- removed unused `filter` param
- bugfix: always set `streamOptions` to `objectMode`
- simplified, converted from push mode (streams 1) to proper pull mode (streams 3)
0.1.0 / 2015-10-25
------------------
- initial release
<!-- contributors -->
[ryanzim]: https://github.com/ryanzim
Generated Vendored
+15
View File
@@ -0,0 +1,15 @@
(The MIT License)
Copyright (c) 2015-2016 JP Richardson
Permission is hereby granted, free of charge, to any person obtaining a copy of this software and associated documentation files
(the 'Software'), to deal in the Software without restriction, including without limitation the rights to use, copy, modify,
merge, publish, distribute, sublicense, and/or sell copies of the Software, and to permit persons to whom the Software is
furnished to do so, subject to the following conditions:
The above copyright notice and this permission notice shall be included in all copies or substantial portions of the Software.
THE SOFTWARE IS PROVIDED 'AS IS', WITHOUT WARRANTY OF ANY KIND, EXPRESS OR IMPLIED, INCLUDING BUT NOT LIMITED TO THE
WARRANTIES OF MERCHANTABILITY, FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE AUTHORS
OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE,
ARISING FROM, OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE SOFTWARE.
+253
View File
@@ -0,0 +1,253 @@
Node.js - klaw
==============
<a href="https://standardjs.com" style="float: right; padding: 0 0 20px 20px;"><img src="https://cdn.rawgit.com/feross/standard/master/sticker.svg" alt="JavaScript Standard Style" width="100" align="right"></a>
A Node.js file system walker extracted from [fs-extra](https://github.com/jprichardson/node-fs-extra).
[![npm Package](https://img.shields.io/npm/v/klaw.svg?style=flat-square)](https://www.npmjs.org/package/klaw)
[![build status](https://api.travis-ci.org/jprichardson/node-klaw.svg)](http://travis-ci.org/jprichardson/node-klaw)
[![windows build status](https://ci.appveyor.com/api/projects/status/github/jprichardson/node-klaw?branch=master&svg=true)](https://ci.appveyor.com/project/jprichardson/node-klaw/branch/master)
Install
-------
npm i --save klaw
If you're using Typescript, we've got [types](https://github.com/DefinitelyTyped/DefinitelyTyped/pull/11492/files):
npm i --save-dev @types/klaw
Name
----
`klaw` is `walk` backwards :p
Sync
----
If you need the same functionality but synchronous, you can use [klaw-sync](https://github.com/manidlou/node-klaw-sync).
Usage
-----
### klaw(directory, [options])
Returns a [Readable stream](https://nodejs.org/api/stream.html#stream_class_stream_readable) that iterates
through every file and directory starting with `dir` as the root. Every `read()` or `data` event
returns an object with two properties: `path` and `stats`. `path` is the full path of the file and
`stats` is an instance of [fs.Stats](https://nodejs.org/api/fs.html#fs_class_fs_stats).
- `directory`: The directory to recursively walk. Type `string`.
- `options`: [Readable stream options](https://nodejs.org/api/stream.html#stream_new_stream_readable_options) and
the following:
- `queueMethod` (`string`, default: `'shift'`): Either `'shift'` or `'pop'`. On `readdir()` array, call either `shift()` or `pop()`.
- `pathSorter` (`function`, default: `undefined`): Sorting [function for Arrays](https://developer.mozilla.org/en-US/docs/Web/JavaScript/Reference/Global_Objects/Array/sort).
- `fs` (`object`, default: [`graceful-fs`](https://github.com/isaacs/node-graceful-fs)): Use this to hook into the `fs` methods or to use [`mock-fs`](https://github.com/tschaub/mock-fs)
- `filter` (`function`, default: `undefined`): Filtering [function for Arrays](https://developer.mozilla.org/en-US/docs/Web/JavaScript/Reference/Global_Objects/Array/filter)
- `depthLimit` (`number`, default: `undefined`): The number of times to recurse before stopping. -1 for unlimited.
- `preserveSymlinks` (`boolean`, default: `false`): Whether symlinks should be followed or treated as items themselves. If true, symlinks will be returned as items in their own right. If false, the linked item will be returned and potentially recursed into, in its stead.
**Streams 1 (push) example:**
```js
const klaw = require('klaw')
const items = [] // files, directories, symlinks, etc
klaw('/some/dir')
.on('data', item => items.push(item.path))
.on('end', () => console.dir(items)) // => [ ... array of files]
```
**Streams 2 & 3 (pull) example:**
```js
const klaw = require('klaw')
const items = [] // files, directories, symlinks, etc
klaw('/some/dir')
.on('readable', function () {
let item
while ((item = this.read())) {
items.push(item.path)
}
})
.on('end', () => console.dir(items)) // => [ ... array of files]
```
### Error Handling
Listen for the `error` event.
Example:
```js
const klaw = require('klaw')
klaw('/some/dir')
.on('readable', function () {
let item
while ((item = this.read())) {
// do something with the file
}
})
.on('error', (err, item) => {
console.log(err.message)
console.log(item.path) // the file the error occurred on
})
.on('end', () => console.dir(items)) // => [ ... array of files]
```
### Aggregation / Filtering / Executing Actions (Through Streams)
On many occasions you may want to filter files based upon size, extension, etc.
Or you may want to aggregate stats on certain file types. Or maybe you want to
perform an action on certain file types.
You should use the module [`through2`](https://www.npmjs.com/package/through2) to easily
accomplish this.
Install `through2`:
npm i --save through2
**Example (skipping directories):**
```js
const klaw = require('klaw')
const through2 = require('through2')
const excludeDirFilter = through2.obj(function (item, enc, next) {
if (!item.stats.isDirectory()) this.push(item)
next()
})
const items = [] // files, directories, symlinks, etc
klaw('/some/dir')
.pipe(excludeDirFilter)
.on('data', item => items.push(item.path))
.on('end', () => console.dir(items)) // => [ ... array of files without directories]
```
**Example (ignore hidden directories):**
```js
const klaw = require('klaw')
const path = require('path')
const filterFunc = item => {
const basename = path.basename(item)
return basename === '.' || basename[0] !== '.'
}
klaw('/some/dir', { filter: filterFunc })
.on('data', item => {
// only items of none hidden folders will reach here
})
```
**Example (totaling size of PNG files):**
```js
const klaw = require('klaw')
const path = require('path')
const through2 = require('through2')
let totalPngsInBytes = 0
const aggregatePngSize = through2.obj(function (item, enc, next) {
if (path.extname(item.path) === '.png') {
totalPngsInBytes += item.stats.size
}
this.push(item)
next()
})
klaw('/some/dir')
.pipe(aggregatePngSize)
.on('data', item => items.push(item.path))
.on('end', () => console.dir(totalPngsInBytes)) // => total of all pngs (bytes)
```
**Example (deleting all .tmp files):**
```js
const fs = require('fs')
const klaw = require('klaw')
const through2 = require('through2')
const deleteAction = through2.obj(function (item, enc, next) {
this.push(item)
if (path.extname(item.path) === '.tmp') {
item.deleted = true
fs.unlink(item.path, next)
} else {
item.deleted = false
next()
}
})
const deletedFiles = []
klaw('/some/dir')
.pipe(deleteAction)
.on('data', item => {
if (!item.deleted) return
deletedFiles.push(item.path)
})
.on('end', () => console.dir(deletedFiles)) // => all deleted files
```
You can even chain a bunch of these filters and aggregators together. By using
multiple pipes.
**Example (using multiple filters / aggregators):**
```js
klaw('/some/dir')
.pipe(filterCertainFiles)
.pipe(deleteSomeOtherFiles)
.on('end', () => console.log('all done!'))
```
**Example passing (piping) through errors:**
Node.js does not `pipe()` errors. This means that the error on one stream, like
`klaw` will not pipe through to the next. If you want to do this, do the following:
```js
const klaw = require('klaw')
const through2 = require('through2')
const excludeDirFilter = through2.obj(function (item, enc, next) {
if (!item.stats.isDirectory()) this.push(item)
next()
})
const items = [] // files, directories, symlinks, etc
klaw('/some/dir')
.on('error', err => excludeDirFilter.emit('error', err)) // forward the error on
.pipe(excludeDirFilter)
.on('data', item => items.push(item.path))
.on('end', () => console.dir(items)) // => [ ... array of files without directories]
```
### Searching Strategy
Pass in options for `queueMethod`, `pathSorter`, and `depthLimit` to affect how the file system
is recursively iterated. See the code for more details, it's less than 50 lines :)
License
-------
MIT
Copyright (c) 2015 [JP Richardson](https://github.com/jprichardson)
+40
View File
@@ -0,0 +1,40 @@
{
"name": "klaw",
"version": "3.0.0",
"description": "File system walker with Readable stream interface.",
"main": "./src/index.js",
"scripts": {
"lint": "standard && standard-markdown",
"test": "npm run lint && npm run unit",
"unit": "tape tests/**/*.js | tap-spec"
},
"repository": {
"type": "git",
"url": "git+https://github.com/jprichardson/node-klaw.git"
},
"keywords": [
"walk",
"walker",
"fs",
"fs-extra",
"readable",
"streams"
],
"author": "JP Richardson",
"license": "MIT",
"bugs": {
"url": "https://github.com/jprichardson/node-klaw/issues"
},
"homepage": "https://github.com/jprichardson/node-klaw#readme",
"dependencies": {
"graceful-fs": "^4.1.9"
},
"devDependencies": {
"mkdirp": "^0.5.1",
"rimraf": "^2.4.3",
"standard": "^11.0.1",
"standard-markdown": "^4.0.1",
"tap-spec": "^5.0.0",
"tape": "^4.2.2"
}
}
+64
View File
@@ -0,0 +1,64 @@
var assert = require('assert')
var path = require('path')
var Readable = require('stream').Readable
var util = require('util')
function Walker (dir, options) {
assert.strictEqual(typeof dir, 'string', '`dir` parameter should be of type string. Got type: ' + typeof dir)
var defaultStreamOptions = { objectMode: true }
var defaultOpts = {
queueMethod: 'shift',
pathSorter: undefined,
filter: undefined,
depthLimit: undefined,
preserveSymlinks: false
}
options = Object.assign(defaultOpts, options, defaultStreamOptions)
Readable.call(this, options)
this.root = path.resolve(dir)
this.paths = [this.root]
this.options = options
if (options.depthLimit > -1) this.rootDepth = this.root.split(path.sep).length + 1
this.fs = options.fs || require('graceful-fs')
}
util.inherits(Walker, Readable)
Walker.prototype._read = function () {
if (this.paths.length === 0) return this.push(null)
var self = this
var pathItem = this.paths[this.options.queueMethod]()
var statFunction = this.options.preserveSymlinks ? self.fs.lstat : self.fs.stat
statFunction(pathItem, function (err, stats) {
var item = { path: pathItem, stats: stats }
if (err) return self.emit('error', err, item)
if (!stats.isDirectory() || (self.rootDepth &&
pathItem.split(path.sep).length - self.rootDepth >= self.options.depthLimit)) {
return self.push(item)
}
self.fs.readdir(pathItem, function (err, pathItems) {
if (err) {
self.push(item)
return self.emit('error', err, item)
}
pathItems = pathItems.map(function (part) { return path.join(pathItem, part) })
if (self.options.filter) pathItems = pathItems.filter(self.options.filter)
if (self.options.pathSorter) pathItems.sort(self.options.pathSorter)
// faster way to do do incremental batch array pushes
self.paths.push.apply(self.paths, pathItems)
self.push(item)
})
})
}
function walk (root, options) {
return new Walker(root, options)
}
module.exports = walk