Welcome to my field notes!

Field notes are notes I leave myself as I go through my day to day work. The hope is that other people will also find these notes useful. Note that these notes are unfiltered and unverified.

Shiny

Useful packages

• shiny - the actual shiny package
• reactloguseful for debugging the reactive graph so we can trace how th inputs go into outputs-
• shinyloadtest - creates realistic traffic on the site so we can mea sure the latency of the app under load.
• shinytest - useful for testing the app before it goes into production.
• promises - for async programming.
• future - for async programming.
• profvis - profiling the performance of the app to find out where the Shiny app may be slow.
• fresh - high level tool to customize Bootstrap 3 or 4.

User Interface

HTML Generation

• Basis of HTML generation in shiny is htmltools
  • tags have some basic attributes (name, attributes, and children)
  • You can modify these using tagAppendAttributes() , tagHasAttribute() and tagGetAttribute().
  • You can set children by using tagSetChildren(), tagAppendChild() or tagAppendChildren()

Dependency Management

• You can directly include in the app (dirty).
• You can use includeCSS() to inline the CSS code into the app (inefficient)
• You can use tags$head(tags$link(rel = "stylesheet", …)) to add the link tag to the head (OK if you are using a CDN, but dirty otherwise)
• Preferred method is to use the htmlDependency constructor and attachDependencies. You use htmlDependency to define where to get the dependency and then use tagList(tag, dependency) to then attach the dependency.
• Benefits are
  • The dependency is only attached once (like in a head tag singleton)
  • You have specific versioning
  • You can work across Windows and UNIX-like systems without any compatibility issues
• resolveDependencies() allows one to only load the specific version for the same dependency!
• findDependencies() on an HTML tag shows all the attached dependencies on a package so that we can use them outside of the original pacakge.
• suppressDependencies() also works for dependencies of other packages (just indicate the name and presumably the package uses the htmlDependency method of injecting dependencies).
• The head argument of the htmlDependency constructor allows one to add arbitrary JSON or JavaScript into the head in a proper way.

Styling the User Interface

• Include via a <link> tag with rel='stylesheet', insert the style in the head directly with a style tag, or insert at the tag level via the style HTML attribute. The preferred is to keep all styling in CSS files since all the other methods are hard to maintain.
• You should use htmlDependency method when possible and attach to a tag so that it’s only accessible.
• Best practices
  • DRY (Don’t Repeat Yourself) - don’t repeat CSS styling, use a CSS class.
  • Keep CSS selectors to the least specific tag, so that .class is usually the first choice.
  • Block Element Modified (BEM) - have self explanatory names for the blocks of the page, like .block__element–modifier. Modifiers are states like disabled, active, etc.
• You can modify CSS on the fly with the Chrome inspector.
• SASS (Syntactically Awesome Style Sheets)
  • Handles code repetition by preprocessing CSS with a preproecessor like Sass, Stylus. There is a sass package which makes it easy to use it in R documents and apps.
  • sass()accepts inputs: (a) R String with CSS, (b) a named list, or (c) a file passed with sass_file() and imports with sass_import().
  • Language features
    • Define variables using $variable: <value> and then use the value with $variable in the CSS declaration.
    • You can define partial CSS files like _partial.css then inline them into the preprocessed files (leading underscore avoids being preprocessed), and then called with @import <filename>.
    • Modules are pieces of Sass files that are later converted into CSS. You can combine all these partials into a module to make it the entrypoint or main CSS script.
      
    • Built in functions allow easy use of the CSS file, like [rgb()](https://rdrr.io/r/grDevices/rgb.html) which returns the hex code for a particular RGB configuration.
    • Inheritance is available as well by defining some common rules via %name-of-set {…} and then using @extend %name-of-set to introduce the rules in that declaration.
    • @if and @else are also available. @each creates a for loop mapping over a vector, and @for allows mapping over a vector that is created for that loop.
  • Best practices
    • group together assets using sass_layer() and then bundle them together using sass_bundle().
    • other developers can use sass_bundle_remove() to manipulate the named layers.
  • Shiny
    • If you add a sass() declaration in the UI, then it is generated every time that the app starts up, but then you need to do that independently if this takes a long time because it adds to the app startup time.
• {fresh} package
  • create_theme() creates a new theme for Bootstrap from Bootswatch.
  • use_theme() is called in the shiny app to create the newly generated theme.
  • bs4Dash
    • bs4dash_status() modifies the status colors
    • bs4dash_color() changes the color configuration
    • search_vars_bs4dash() lists down the variables that can be used and their default values.
    • supply the theme to the freshTheme argument of the dashboardPage.
  • {shinydashboard}
    • Create using adminlte_*() functions then pass to customize_shinydashboard to then insert into the UI function
• bslib package (requires {shiny} >= 1.6)
  • this is a package built by RStudio. It requires changes to be able to support and currently does not support {shinydashboard} and {bs4dash}.
  • Create themes using bs_theme(), update them with bs_update_theme() and preview them with bs_theme_preview(), you can pass with_themer = TRUE to have a live preview. bs_add_rules() allows you to add arbitrary CSS rules.
  • session$setCurrentTheme() allows one to change the theme dynamically.
  • bs_dependency() allows one to define a sass dependency that is dependent on the variables defined in the main bootstrap theme. (You can create a fallback by conditioning on is_bs_theme() and then use bs_dependency_defer() to dynamically generate the dependency.
  • Add run_with_themer() to enable the running of the dynamic themer.

JavaScript

• A knowledge of JavaScript will go a long way to ensuring that we can produce custom interactions in a Shiny app.
• Shiny is designed for a high degree of bi-directional transfer, so traditional HTTP is not a choice. Shiny instead uses websockets3. {httpuv} provides the websocker server and {websockers} and/or direct JS create the client.
  • httpuv can handle via callbacks such as onMessage() and onClose() and then send messages via send(). {websockets} also uses these semantics on the client side.
  • instantiated via the CONNECT http verb.
  • most usually the websocket client is handled directly by JavaScript because it is faster and more performant.
  • There are handlers on the R and JS side that handle input and output as seen in Shiny.
  • Note that there is no concurrency on the R side as all websocket handles will be queued.
• Shiny (R) and JS communicate via JSON.
• session object
  • this is the main identifier for a particular client connection in shiny.
  • sendCustomMessage() sends messages from R to JS
    • Shiny.AddCustomMessageHandler(<message_type>, function(message) {}) then hanldes this on the JavaScript side.
    • This is ideal compared to renderUI when doing complex manipulations as we avoid having to re-render the UI repeatedly.
  • sendInputMessage() updates inputs from the server
• Shiny events documentation

Input System

Input bindings

• Input bindings bind the input to sending a message to R whenever something changes in the input.
• Input tag (<input>) structure
  • id is the main identifier and is supposed to be globally unique.
  • type is the type of the input
  • class may be required to find the element in the DOM and for styling
  • value holds the current input value.
• Input binding functions
  • [find(scope)](https://rdrr.io/r/utils/apropos.html) finds all the instances of that input binding in the scope DOM element. The R UI function should generate HTML tags that can be found by this JS function, and also includes this htmlDependency. Use jQuery.
  • initialize(el) (optional) - some JS/HTML/CSS frameworks require some initialization function to run before it can be functional. Put it here. Shiny avoids this being called more than once.
  • getValue(el) - returns the input value as a JSON string. Don’t forget the return statement.
  • setValue(el, value) (optional) - This is only necessary if we need to update the value of the input from R. You will still need to apply an updateInput() R function and a receiveMessage() equivalent that call this though.
  • receiveMessage(el, data) (optional) - triggered when the session$sendInputMessage(inputId, message) method is called from R. If data.value argument has property then you can interpret this and send it to this.setValue(). Always trigger $(el).trigger('change') inside when you want the new input value to propagate.
  • subscribe(el) - subscribes to events that would update the input. (use jQuery $(el).on('<event>', <callback>)). You need to call callback(true) inside the callback in order to make Shiny capture it. When you want to also react when updateInput() is called then you will need to put callback(false) in the function.
  • getRatePolicy() - allows debouncing4 and returns an object with two elements
    • policy - "direct", "debounce", and "throttle" debounce delays unitl nothing is received for delay, and throttle only sends every delay seconds
    • delay - number indicating the number of milliseconds
    • Only applied when callback() is called with true
  • Miscelleanous and optional, not typically changed:
    • getId() - returns the object ID, otherwise takes the default one in the InputBinding class
    • getType() - gets the type to handle the custom data format.
    • getState()
• You can edit an input binding, by unbindAll(), then $.extend(Shiny.inputBindings.bindingNames['PACKAGE_NAME.BINDING_NAME'].binding, {}, then calling Shiny.bindAll() again.
• You can update the binding from the client to save a roundtrip to the server.
• Secondary inputs
  • Not designed to be inputs, but we can cause them to cause events to fire to the server via the Shiny input system. For example capture if a box is maximized or minimized, or do that from the R function.
  • This method is more efficient than renderUI as it only does what is necessary.
  • The setValue(el, value) method can receive a value.action that does different things based on what we sent.
  • You can add internal methods to the binding using _function that can be called inside all the Shiny inherited functions.
• Shiny.setInputValue("<input_id>", value, {priority: "event"}); sets the input value even if there are not changes.
• Input handlers
  • On the R side, registerInputHandler("binding_id", function(data, …) {}) allows you to parse anything that comes from that input.
  • Another way is to leverage getType() and use the built in handler for {shiny}.

Shiny input lifecycle

• Initialization
  • Bind all inputs and outputs with bindAll()
  • Initialize all inputs
  • Initialize the websocket connection and send the initial values to the server.
  • For each binding in the binding registry, register it by find ing all instances, and then trigger getId() to find the ID, getType() to register the input handler, getValue() gets the initial value, and subscribe registers even listeners driving the input behavior.
  • The shiny-input-binding attribute is added, the shiny-bound-input class is added, and triggers the shiny:bound event.
  • shiny:connected is triggered
  • shiny:sessioninitialized is triggered when all the values are sent to R.

Inner Workings

Libraries

• Bootstrap - very popular frontend CSS framework that offers responsiveness and easy to use components
• jQuery - for DOM manipulation.
• Suppress default dependencies using suppressDependencies() . You can also use this to suppress dependencies from other packages.
• It’s located in the inst/www/shared directory of the shiny package source.

Lifecycle

1. Client sends HTTP CONNECT request to establish the connection
2. runApp() is called:
   1. shinyApp() is called to instantiate the app object inside R memory
   2. startApp() creates HTTP and websocket (WS) handlers to allow communication between R and JavaScript.
   3. startServer() from {httpuv} starts the server and opens the server websocket connection.
3. If the R code has no errors, then the Shiny HTML generated is sent to the client.
   1. R code becomes HTML through the [htmltools::renderDocument](https://rdrr.io/pkg/htmltools/man/renderDocument.html) function and shiny:::renderPage().
      1. Convert all the R tags to HTML with renderTags(). There is a head content, singleton, dependencies, and HTML.
      2. The dependencies are treated with resolveDependencies to avoid conflicts.
      3. The dependencies are processed with createWebDependency which makes sure the dependency can be served with HTTP.
      4. renderDependencies inserts the dependency code inside the head tag of the HTML.
   2. Send the HTML string as a response to the client via shiny::httpResponse()
4. The HTML code is received and interpreted by the web browser.
5. The HTML page is rendered. This HTML and JS contains all the necessary JS code to establish the websocket connection back to R.

Miscellaneous

• Query strings - parseQueryString() parses the query parameter and can be updated using updateQueryString()
• Licensing - (not legal advice) AGPL does not actually hinder you from deploying commercial services using shiny server, only from selling or modifying the shiny server code itself.