|
||||||||||
| PREV CLASS NEXT CLASS | FRAMES NO FRAMES | |||||||||
| SUMMARY: NESTED | FIELD | CONSTR | METHOD | DETAIL: FIELD | CONSTR | METHOD | |||||||||
java.lang.Objectorg.apache.hadoop.fs.AbstractFileSystem
org.apache.hadoop.fs.viewfs.ViewFs
@InterfaceAudience.Public @InterfaceStability.Evolving public class ViewFs
ViewFs (extends the AbstractFileSystem interface) implements a client-side mount table. The viewFs file system is implemented completely in memory on the client side. The client-side mount table allows a client to provide a customized view of a file system namespace that is composed from one or more individual file systems (a localFs or Hdfs, S3fs, etc). For example one could have a mount table that provides links such as
To use viewfs one would typically set the default file system in the config (i.e. fs.default.name< = viewfs:///) along with the mount table config variables as described below.
** Config variables to specify the mount table entries **
The file system is initialized from the standard Hadoop config through
config variables.
See FsConstants for URI and Scheme constants;
See Constants for config var constants;
see ConfigUtil for convenient lib.
All the mount table config entries for view fs are prefixed by fs.viewfs.mounttable. For example the above example can be specified with the following config variables:
**** Merge Mounts **** (NOTE: merge mounts are not implemented yet.)
One can also use "MergeMounts" to merge several directories (this is sometimes called union-mounts or junction-mounts in the literature. For example of the home directories are stored on say two file systems (because they do not fit on one) then one could specify a mount entry such as following merges two dirs:
| Field Summary |
|---|
| Fields inherited from class org.apache.hadoop.fs.AbstractFileSystem |
|---|
statistics |
| Constructor Summary | |
|---|---|
ViewFs(Configuration conf)
|
|
| Method Summary | |
|---|---|
void |
access(Path path,
FsAction mode)
The specification of this method matches that of FileContext.access(Path, FsAction)
except that an UnresolvedLinkException may be thrown if a symlink is
encountered in the path. |
FSDataOutputStream |
createInternal(Path f,
EnumSet<CreateFlag> flag,
FsPermission absolutePermission,
int bufferSize,
short replication,
long blockSize,
Progressable progress,
org.apache.hadoop.fs.Options.ChecksumOpt checksumOpt,
boolean createParent)
The specification of this method matches that of AbstractFileSystem.create(Path, EnumSet, Options.CreateOpts...) except that the opts
have been declared explicitly. |
void |
createSymlink(Path target,
Path link,
boolean createParent)
The specification of this method matches that of FileContext.createSymlink(Path, Path, boolean); |
boolean |
delete(Path f,
boolean recursive)
The specification of this method matches that of FileContext.delete(Path, boolean) except that Path f must be for
this file system. |
AclStatus |
getAclStatus(Path path)
Gets the ACLs of files and directories. |
List<org.apache.hadoop.security.token.Token<?>> |
getDelegationTokens(String renewer)
Get one or more delegation tokens associated with the filesystem. |
BlockLocation[] |
getFileBlockLocations(Path f,
long start,
long len)
The specification of this method matches that of FileContext.getFileBlockLocations(Path, long, long) except that
Path f must be for this file system. |
FileChecksum |
getFileChecksum(Path f)
The specification of this method matches that of FileContext.getFileChecksum(Path) except that Path f must be for
this file system. |
FileStatus |
getFileLinkStatus(Path f)
The specification of this method matches that of FileContext.getFileLinkStatus(Path)
except that an UnresolvedLinkException may be thrown if a symlink is
encountered in the path leading up to the final path component. |
FileStatus |
getFileStatus(Path f)
The specification of this method matches that of FileContext.getFileStatus(Path)
except that an UnresolvedLinkException may be thrown if a symlink is
encountered in the path. |
FsStatus |
getFsStatus()
The specification of this method matches that of FileContext.getFsStatus(Path). |
Path |
getHomeDirectory()
Return the current user's home directory in this file system. |
Path |
getLinkTarget(Path f)
Partially resolves the path. |
org.apache.hadoop.fs.viewfs.ViewFs.MountPoint[] |
getMountPoints()
|
FsServerDefaults |
getServerDefaults()
Return a set of server default configuration values. |
int |
getUriDefaultPort()
The default port of this file system. |
byte[] |
getXAttr(Path path,
String name)
Get an xattr for a file or directory. |
Map<String,byte[]> |
getXAttrs(Path path)
Get all of the xattrs for a file or directory. |
Map<String,byte[]> |
getXAttrs(Path path,
List<String> names)
Get all of the xattrs for a file or directory. |
boolean |
isValidName(String src)
Returns true if the specified string is considered valid in the path part of a URI by this file system. |
FileStatus[] |
listStatus(Path f)
The specification of this method matches that of FileContext.Util.listStatus(Path) except that Path f must be
for this file system. |
org.apache.hadoop.fs.RemoteIterator<FileStatus> |
listStatusIterator(Path f)
The specification of this method matches that of FileContext.listStatus(Path) except that Path f must be for this
file system. |
List<String> |
listXAttrs(Path path)
Get all of the xattr names for a file or directory. |
void |
mkdir(Path dir,
FsPermission permission,
boolean createParent)
The specification of this method matches that of FileContext.mkdir(Path, FsPermission, boolean) except that the Path
f must be fully qualified and the permission is absolute (i.e. |
void |
modifyAclEntries(Path path,
List<AclEntry> aclSpec)
Modifies ACL entries of files and directories. |
FSDataInputStream |
open(Path f,
int bufferSize)
The specification of this method matches that of FileContext.open(Path, int) except that Path f must be for this
file system. |
void |
removeAcl(Path path)
Removes all but the base ACL entries of files and directories. |
void |
removeAclEntries(Path path,
List<AclEntry> aclSpec)
Removes ACL entries from files and directories. |
void |
removeDefaultAcl(Path path)
Removes all default ACL entries from files and directories. |
void |
removeXAttr(Path path,
String name)
Remove an xattr of a file or directory. |
void |
renameInternal(Path src,
Path dst)
The specification of this method matches that of FileContext.rename(Path, Path, Options.Rename...) except that Path
f must be for this file system and NO OVERWRITE is performed. |
void |
renameInternal(Path src,
Path dst,
boolean overwrite)
The specification of this method matches that of FileContext.rename(Path, Path, Options.Rename...) except that Path
f must be for this file system. |
Path |
resolvePath(Path f)
Return the fully-qualified path of path f resolving the path through any internal symlinks or mount point |
void |
setAcl(Path path,
List<AclEntry> aclSpec)
Fully replaces ACL of files and directories, discarding all existing entries. |
void |
setOwner(Path f,
String username,
String groupname)
The specification of this method matches that of FileContext.setOwner(Path, String, String) except that Path f must
be for this file system. |
void |
setPermission(Path f,
FsPermission permission)
The specification of this method matches that of FileContext.setPermission(Path, FsPermission) except that Path f
must be for this file system. |
boolean |
setReplication(Path f,
short replication)
The specification of this method matches that of FileContext.setReplication(Path, short) except that Path f must be
for this file system. |
void |
setTimes(Path f,
long mtime,
long atime)
The specification of this method matches that of FileContext.setTimes(Path, long, long) except that Path f must be
for this file system. |
void |
setVerifyChecksum(boolean verifyChecksum)
The specification of this method matches that of FileContext.setVerifyChecksum(boolean, Path) except that Path f
must be for this file system. |
void |
setXAttr(Path path,
String name,
byte[] value,
EnumSet<XAttrSetFlag> flag)
Set an xattr of a file or directory. |
boolean |
supportsSymlinks()
Returns true if the file system supports symlinks, false otherwise. |
| Methods inherited from class org.apache.hadoop.fs.AbstractFileSystem |
|---|
checkPath, checkScheme, clearStatistics, create, createFileSystem, equals, get, getAllStatistics, getCanonicalServiceName, getFsStatus, getInitialWorkingDirectory, getStatistics, getStatistics, getUri, getUriPath, hashCode, listCorruptFileBlocks, listLocatedStatus, makeQualified, open, printStatistics, rename, setXAttr |
| Methods inherited from class java.lang.Object |
|---|
clone, finalize, getClass, notify, notifyAll, toString, wait, wait, wait |
| Constructor Detail |
|---|
public ViewFs(Configuration conf)
throws IOException,
URISyntaxException
IOException
URISyntaxException| Method Detail |
|---|
public FsServerDefaults getServerDefaults()
throws IOException
AbstractFileSystem
getServerDefaults in class AbstractFileSystemIOException - an I/O error occurredpublic int getUriDefaultPort()
AbstractFileSystem
getUriDefaultPort in class AbstractFileSystempublic Path getHomeDirectory()
AbstractFileSystem
getHomeDirectory in class AbstractFileSystem
public Path resolvePath(Path f)
throws FileNotFoundException,
org.apache.hadoop.security.AccessControlException,
org.apache.hadoop.fs.UnresolvedLinkException,
IOException
AbstractFileSystem
resolvePath in class AbstractFileSystemf - path to be resolved
FileNotFoundException
org.apache.hadoop.security.AccessControlException
org.apache.hadoop.fs.UnresolvedLinkException
IOException
public FSDataOutputStream createInternal(Path f,
EnumSet<CreateFlag> flag,
FsPermission absolutePermission,
int bufferSize,
short replication,
long blockSize,
Progressable progress,
org.apache.hadoop.fs.Options.ChecksumOpt checksumOpt,
boolean createParent)
throws org.apache.hadoop.security.AccessControlException,
FileAlreadyExistsException,
FileNotFoundException,
ParentNotDirectoryException,
UnsupportedFileSystemException,
org.apache.hadoop.fs.UnresolvedLinkException,
IOException
AbstractFileSystemAbstractFileSystem.create(Path, EnumSet, Options.CreateOpts...) except that the opts
have been declared explicitly.
createInternal in class AbstractFileSystemorg.apache.hadoop.security.AccessControlException
FileAlreadyExistsException
FileNotFoundException
ParentNotDirectoryException
UnsupportedFileSystemException
org.apache.hadoop.fs.UnresolvedLinkException
IOException
public boolean delete(Path f,
boolean recursive)
throws org.apache.hadoop.security.AccessControlException,
FileNotFoundException,
org.apache.hadoop.fs.UnresolvedLinkException,
IOException
AbstractFileSystemFileContext.delete(Path, boolean) except that Path f must be for
this file system.
delete in class AbstractFileSystemorg.apache.hadoop.security.AccessControlException
FileNotFoundException
org.apache.hadoop.fs.UnresolvedLinkException
IOException
public BlockLocation[] getFileBlockLocations(Path f,
long start,
long len)
throws org.apache.hadoop.security.AccessControlException,
FileNotFoundException,
org.apache.hadoop.fs.UnresolvedLinkException,
IOException
AbstractFileSystemFileContext.getFileBlockLocations(Path, long, long) except that
Path f must be for this file system.
getFileBlockLocations in class AbstractFileSystemorg.apache.hadoop.security.AccessControlException
FileNotFoundException
org.apache.hadoop.fs.UnresolvedLinkException
IOException
public FileChecksum getFileChecksum(Path f)
throws org.apache.hadoop.security.AccessControlException,
FileNotFoundException,
org.apache.hadoop.fs.UnresolvedLinkException,
IOException
AbstractFileSystemFileContext.getFileChecksum(Path) except that Path f must be for
this file system.
getFileChecksum in class AbstractFileSystemorg.apache.hadoop.security.AccessControlException
FileNotFoundException
org.apache.hadoop.fs.UnresolvedLinkException
IOException
public FileStatus getFileStatus(Path f)
throws org.apache.hadoop.security.AccessControlException,
FileNotFoundException,
org.apache.hadoop.fs.UnresolvedLinkException,
IOException
AbstractFileSystemFileContext.getFileStatus(Path)
except that an UnresolvedLinkException may be thrown if a symlink is
encountered in the path.
getFileStatus in class AbstractFileSystemorg.apache.hadoop.security.AccessControlException
FileNotFoundException
org.apache.hadoop.fs.UnresolvedLinkException
IOException
public void access(Path path,
FsAction mode)
throws org.apache.hadoop.security.AccessControlException,
FileNotFoundException,
org.apache.hadoop.fs.UnresolvedLinkException,
IOException
AbstractFileSystemFileContext.access(Path, FsAction)
except that an UnresolvedLinkException may be thrown if a symlink is
encountered in the path.
org.apache.hadoop.security.AccessControlException
FileNotFoundException
org.apache.hadoop.fs.UnresolvedLinkException
IOException
public FileStatus getFileLinkStatus(Path f)
throws org.apache.hadoop.security.AccessControlException,
FileNotFoundException,
UnsupportedFileSystemException,
IOException
AbstractFileSystemFileContext.getFileLinkStatus(Path)
except that an UnresolvedLinkException may be thrown if a symlink is
encountered in the path leading up to the final path component.
If the file system does not support symlinks then the behavior is
equivalent to AbstractFileSystem.getFileStatus(Path).
getFileLinkStatus in class AbstractFileSystemorg.apache.hadoop.security.AccessControlException
FileNotFoundException
UnsupportedFileSystemException
IOException
public FsStatus getFsStatus()
throws org.apache.hadoop.security.AccessControlException,
FileNotFoundException,
IOException
AbstractFileSystemFileContext.getFsStatus(Path).
getFsStatus in class AbstractFileSystemorg.apache.hadoop.security.AccessControlException
FileNotFoundException
IOException
public org.apache.hadoop.fs.RemoteIterator<FileStatus> listStatusIterator(Path f)
throws org.apache.hadoop.security.AccessControlException,
FileNotFoundException,
org.apache.hadoop.fs.UnresolvedLinkException,
IOException
AbstractFileSystemFileContext.listStatus(Path) except that Path f must be for this
file system.
listStatusIterator in class AbstractFileSystemorg.apache.hadoop.security.AccessControlException
FileNotFoundException
org.apache.hadoop.fs.UnresolvedLinkException
IOException
public FileStatus[] listStatus(Path f)
throws org.apache.hadoop.security.AccessControlException,
FileNotFoundException,
org.apache.hadoop.fs.UnresolvedLinkException,
IOException
AbstractFileSystemFileContext.Util.listStatus(Path) except that Path f must be
for this file system.
listStatus in class AbstractFileSystemorg.apache.hadoop.security.AccessControlException
FileNotFoundException
org.apache.hadoop.fs.UnresolvedLinkException
IOException
public void mkdir(Path dir,
FsPermission permission,
boolean createParent)
throws org.apache.hadoop.security.AccessControlException,
FileAlreadyExistsException,
FileNotFoundException,
org.apache.hadoop.fs.UnresolvedLinkException,
IOException
AbstractFileSystemFileContext.mkdir(Path, FsPermission, boolean) except that the Path
f must be fully qualified and the permission is absolute (i.e.
umask has been applied).
mkdir in class AbstractFileSystemorg.apache.hadoop.security.AccessControlException
FileAlreadyExistsException
FileNotFoundException
org.apache.hadoop.fs.UnresolvedLinkException
IOException
public FSDataInputStream open(Path f,
int bufferSize)
throws org.apache.hadoop.security.AccessControlException,
FileNotFoundException,
org.apache.hadoop.fs.UnresolvedLinkException,
IOException
AbstractFileSystemFileContext.open(Path, int) except that Path f must be for this
file system.
open in class AbstractFileSystemorg.apache.hadoop.security.AccessControlException
FileNotFoundException
org.apache.hadoop.fs.UnresolvedLinkException
IOException
public void renameInternal(Path src,
Path dst,
boolean overwrite)
throws IOException,
org.apache.hadoop.fs.UnresolvedLinkException
AbstractFileSystemFileContext.rename(Path, Path, Options.Rename...) except that Path
f must be for this file system.
renameInternal in class AbstractFileSystemIOException
org.apache.hadoop.fs.UnresolvedLinkException
public void renameInternal(Path src,
Path dst)
throws org.apache.hadoop.security.AccessControlException,
FileAlreadyExistsException,
FileNotFoundException,
ParentNotDirectoryException,
org.apache.hadoop.fs.UnresolvedLinkException,
IOException
AbstractFileSystemFileContext.rename(Path, Path, Options.Rename...) except that Path
f must be for this file system and NO OVERWRITE is performed.
File systems that do not have a built in overwrite need implement only this
method and can take advantage of the default impl of the other
AbstractFileSystem.renameInternal(Path, Path, boolean)
renameInternal in class AbstractFileSystemorg.apache.hadoop.security.AccessControlException
FileAlreadyExistsException
FileNotFoundException
ParentNotDirectoryException
org.apache.hadoop.fs.UnresolvedLinkException
IOExceptionpublic boolean supportsSymlinks()
AbstractFileSystem
supportsSymlinks in class AbstractFileSystem
public void createSymlink(Path target,
Path link,
boolean createParent)
throws IOException,
org.apache.hadoop.fs.UnresolvedLinkException
AbstractFileSystemFileContext.createSymlink(Path, Path, boolean);
createSymlink in class AbstractFileSystemIOException
org.apache.hadoop.fs.UnresolvedLinkException
public Path getLinkTarget(Path f)
throws IOException
AbstractFileSystemFSLinkResolver, and differs from the similarly named method
FileContext.getLinkTarget(Path).
getLinkTarget in class AbstractFileSystemIOException - subclass implementations may throw IOException
public void setOwner(Path f,
String username,
String groupname)
throws org.apache.hadoop.security.AccessControlException,
FileNotFoundException,
org.apache.hadoop.fs.UnresolvedLinkException,
IOException
AbstractFileSystemFileContext.setOwner(Path, String, String) except that Path f must
be for this file system.
setOwner in class AbstractFileSystemorg.apache.hadoop.security.AccessControlException
FileNotFoundException
org.apache.hadoop.fs.UnresolvedLinkException
IOException
public void setPermission(Path f,
FsPermission permission)
throws org.apache.hadoop.security.AccessControlException,
FileNotFoundException,
org.apache.hadoop.fs.UnresolvedLinkException,
IOException
AbstractFileSystemFileContext.setPermission(Path, FsPermission) except that Path f
must be for this file system.
setPermission in class AbstractFileSystemorg.apache.hadoop.security.AccessControlException
FileNotFoundException
org.apache.hadoop.fs.UnresolvedLinkException
IOException
public boolean setReplication(Path f,
short replication)
throws org.apache.hadoop.security.AccessControlException,
FileNotFoundException,
org.apache.hadoop.fs.UnresolvedLinkException,
IOException
AbstractFileSystemFileContext.setReplication(Path, short) except that Path f must be
for this file system.
setReplication in class AbstractFileSystemorg.apache.hadoop.security.AccessControlException
FileNotFoundException
org.apache.hadoop.fs.UnresolvedLinkException
IOException
public void setTimes(Path f,
long mtime,
long atime)
throws org.apache.hadoop.security.AccessControlException,
FileNotFoundException,
org.apache.hadoop.fs.UnresolvedLinkException,
IOException
AbstractFileSystemFileContext.setTimes(Path, long, long) except that Path f must be
for this file system.
setTimes in class AbstractFileSystemorg.apache.hadoop.security.AccessControlException
FileNotFoundException
org.apache.hadoop.fs.UnresolvedLinkException
IOException
public void setVerifyChecksum(boolean verifyChecksum)
throws org.apache.hadoop.security.AccessControlException,
IOException
AbstractFileSystemFileContext.setVerifyChecksum(boolean, Path) except that Path f
must be for this file system.
setVerifyChecksum in class AbstractFileSystemorg.apache.hadoop.security.AccessControlException
IOExceptionpublic org.apache.hadoop.fs.viewfs.ViewFs.MountPoint[] getMountPoints()
public List<org.apache.hadoop.security.token.Token<?>> getDelegationTokens(String renewer)
throws IOException
AbstractFileSystem
renewer - the account name that is allowed to renew the token.
IOExceptionpublic boolean isValidName(String src)
AbstractFileSystem
isValidName in class AbstractFileSystemsrc - String source filename to check, path part of the URI
public void modifyAclEntries(Path path,
List<AclEntry> aclSpec)
throws IOException
AbstractFileSystem
modifyAclEntries in class AbstractFileSystempath - Path to modifyaclSpec - ListIOException - if an ACL could not be modified
public void removeAclEntries(Path path,
List<AclEntry> aclSpec)
throws IOException
AbstractFileSystem
removeAclEntries in class AbstractFileSystempath - Path to modifyaclSpec - ListIOException - if an ACL could not be modified
public void removeDefaultAcl(Path path)
throws IOException
AbstractFileSystem
removeDefaultAcl in class AbstractFileSystempath - Path to modify
IOException - if an ACL could not be modified
public void removeAcl(Path path)
throws IOException
AbstractFileSystem
removeAcl in class AbstractFileSystempath - Path to modify
IOException - if an ACL could not be removed
public void setAcl(Path path,
List<AclEntry> aclSpec)
throws IOException
AbstractFileSystem
setAcl in class AbstractFileSystempath - Path to modifyaclSpec - ListIOException - if an ACL could not be modified
public AclStatus getAclStatus(Path path)
throws IOException
AbstractFileSystem
getAclStatus in class AbstractFileSystempath - Path to get
IOException - if an ACL could not be read
public void setXAttr(Path path,
String name,
byte[] value,
EnumSet<XAttrSetFlag> flag)
throws IOException
AbstractFileSystem
setXAttr in class AbstractFileSystempath - Path to modifyname - xattr name.value - xattr value.flag - xattr set flag
IOException
public byte[] getXAttr(Path path,
String name)
throws IOException
AbstractFileSystem
getXAttr in class AbstractFileSystempath - Path to get extended attributename - xattr name.
IOException
public Map<String,byte[]> getXAttrs(Path path)
throws IOException
AbstractFileSystem
getXAttrs in class AbstractFileSystempath - Path to get extended attributes
IOException
public Map<String,byte[]> getXAttrs(Path path,
List<String> names)
throws IOException
AbstractFileSystem
getXAttrs in class AbstractFileSystempath - Path to get extended attributesnames - XAttr names.
IOException
public List<String> listXAttrs(Path path)
throws IOException
AbstractFileSystem
listXAttrs in class AbstractFileSystempath - Path to get extended attributes
IOException
public void removeXAttr(Path path,
String name)
throws IOException
AbstractFileSystem
removeXAttr in class AbstractFileSystempath - Path to remove extended attributename - xattr name
IOException
|
||||||||||
| PREV CLASS NEXT CLASS | FRAMES NO FRAMES | |||||||||
| SUMMARY: NESTED | FIELD | CONSTR | METHOD | DETAIL: FIELD | CONSTR | METHOD | |||||||||