File Systems

Table of Contents

• Overview
• Local File System
• Dropbox File System

Overview

File systems abstraction allows you to access transparently different type of storages, either it is local file, AWS S3, or DropBox. The file system type is specified via URL in scheme place, i.e. to specify local file use file:// on scheme place and then path to the file like /my/directory/file.txt, so the URL of the file would look like file:///my/directory/file.txt.

By default only local file is accessible but you can always add more. Currently you would need to register file system driver yourself if it’s different from local file system. To do that call appropriate method on io.wavebeans.fs.core.WbFileDriver object:

// register driver, you can only do that once for specific scheme
WbFileDriver.registerDriver("myscheme", MyDriverImplementation())
// unregister driver, so you can register another one if you need
WbFileDriver.unregisterDriver("myscheme")

Driver implements the io.wavebeans.fs.core.WbFileDriver interface.

Usually you won’t need to access file system directly, you would always use methods responsible to call the file system themselves, i.e. wav("file:///my/directory/file.wav"). But here some tips if you ever need to:

// instantiate the driver of certain type
val fileDriver = WbFileDriver.instance("file")
// create a temporary file, it'll generate random filler to make sure the file is new
val temporaryFile = fileDriver.createTemporaryWbFile("temp", "txt")
// create a file pointer by URI
val file = fileDriver.createWbFile(URI("file:///my/directory/file.txt"))
// shortcut to create the file pointer of certain scheme
/*val file = */WbFileDriver.createFile(URI("file:///my/directory/file.txt"))
// check if file exists
file.exists()
// delete file
file.delete()
// create output stream to write file content
file.createWbFileOutputStream()
// create input stream to read file content
file.createWbFileInputStream()

Overall follow code-level API documentation.

Local File System

Local File System is supported by default and you don’t need to do something special in order to use it. Just be aware, that in distributed mode it won’t work as expected.

To use the local file system, specify the file scheme in the URI. Also you need to be aware that only absolute path is supported at this moment, so every time you need to specify the path from the root. It might be handy to use java.io.File and resolve the absolute path using its API:

val inputFile = File("input.wav")
val input = wave("file://${inputFile.absolutePath}")

Dropbox File System

DropBox File System allows work with files in your DropBox folder. It’s a little tricky to configure and requires a little more effort, but overall quite handy.

To start working with your DropBox account you’ll need to go to its Developer console and create the key:

1. Go to Dropbox App Console: https://www.dropbox.com/developers/apps
2. Click “Create app” button.
3. Choose an API: “Dropbox API”.
4. Choose the type of access: “Full Dropbox” or “App folder”. Doesn’t matter. Though the second option is a little safer overall, but it is up to you and your requirements.
5. Name your app, remember the name you’ve given. Later on it’ll be used as “Client Identifier”.
6. Hit “Create app” button.
7. On “Settings” tab generate a new “Access token” and copy it. You’ll need it later.

Dropbox filesystem distributed as a separate library, so add it as a dependency, i.e. for gradle:

dependencies {
   implementation "io.wavebeans:filesystems-dropbox:$wavebeans_version"
}

Then you need to configure the driver. It’ll automatically reigster itself under dropbox scheme:

DropboxWbFileDriver.configure(
        "Client identifier",
        "Access Token"
)

After that action you may start using files via dropbox scheme:

val input = wave("dropbox:///folder/input.wav")

If you want to register the driver yourself, for instance under different name, or use a few drivers at a time, you may just instantiate the object of class io.wavebeans.fs.dropbox.DropboxWbFileDriver and register it via call to WbFileDriver.registerDriver().

Production usage

The approach provided above is handy for running server for sole use or testing, however for multi-user or user-agnostic deployment can’t be used. To get a proper access token you would need to follow the OAuth flow which is perfectly explained in Dropox developer’s documentation. Though here are some snippets based on Java Dropbox SDK com.dropbox.core:dropbox-core-sdk:3.1.4.

1. Initialize auth client.

    val auth = DbxWebAuth(
            DbxRequestConfig("<Client Identifier>"),
            DbxAppInfo("<Client Key>", "<Client Secret>")
    )
   

2. You would need to create a session storage, you may use com.dropbox.core.DbxStandardSessionStore if you’re running some servlet container web server like Tomcat or Jetty. But here for the sake of simplicity we’ll use some simple implementation based on file:

    val sessionStore = object : DbxSessionStore {
       
        private val csrf = File("/some/directory/csrf.code")
       
        override fun clear() {
            csrf.createNewFile()
        }
       
        override fun get(): String = csrf.readText()
       
        override fun set(value: String) {
            csrf.createNewFile()
            csrf.writeText(value)
        }
       
    }
   

3. Define a return URL, we’ll need it exactly in that form in two places. Specify any path or any set of parameters you require. Also don’t forget to register it in Dropbox App console.

    val ruri = "http://localhost:8888"
   

4. Generate authorization URL and redirect user there.

    val uri = auth.authorize(DbxWebAuth.Request.newBuilder()
            .withRedirectUri(ruri, sessionStore)
            .build())
       
    redirectTo(uri)
   

5. When user returned back on your return URL, in the handler extract state and code out of query parameters and request for access token, which later on you’ll use to access the Dropbox API.

    val state = queryParameters["state"]
    val code = queryParameters["code"]
    val accessToken = auth.finishFromRedirect(
            ruri, 
            sessionStore,
            mapOf(
                    "state" to arrayOf(state),
                    "code" to arrayOf(code)
            )
    ).accessToken
   

6. Test it out.

    DropboxWbFileDriver.configure(
            "<Client Identifier>",
            accessToken
    )
       
    WbFileDriver.createFile(URI("dropbox:///test.txt")).createWbFileOutputStream().writer().use {
        it.write("Hello world!")
    }