You are currently browsing jjallaire’s articles.

The dygraphs package is an R interface to the dygraphs JavaScript charting library. It provides rich facilities for charting time-series data in R, including:

  • Automatically plots xts time-series objects (or objects convertible to xts).
  • Rich interactive features including zoom/pan and series/point highlighting.
  • Highly configurable axis and series display (including optional 2nd Y-axis).
  • Display upper/lower bars (e.g. prediction intervals) around series.
  • Various graph overlays including shaded regions, event lines, and annotations.
  • Use at the R console just like conventional R plots (via RStudio Viewer).
  • Embeddable within R Markdown documents and Shiny web applications.

The dygraphs package is available on CRAN now and can be installed with:

install.packages("dygraphs")

Examples

Here are some examples of interactive time series visualizations you can create with only a line or two of R code (the screenshots are static, click them to see the interactive version).

Panning and Zooming

This code adds a range selector that’s can be used to pan and zoom around the series data:

dygraph(nhtemp, main = "New Haven Temperatures") %>%
  dyRangeSelector()

Screen Shot 2015-04-09 at 1.01.35 PM

Point Highlighting

When you hover over the time-series the values of all points at the location of the mouse are shown in the legend:

lungDeaths <- cbind(ldeaths, mdeaths, fdeaths)
dygraph(lungDeaths, main = "Deaths from Lung Disease (UK)") %>%
  dyOptions(colors = RColorBrewer::brewer.pal(3, "Set2"))

Screen Shot 2015-04-09 at 12.53.54 PM

Shading and Annotations

There are a wide variety of tools available to annotate time series. Here we demonstrate creating shaded regions:

dygraph(nhtemp, main="New Haven Temperatures") %>% 
  dySeries(label="Temp (F)", color="black") %>%
  dyShading(from="1920-1-1", to="1930-1-1", color="#FFE6E6") %>%
  dyShading(from="1940-1-1", to="1950-1-1", color="#CCEBD6")

Screen Shot 2015-04-09 at 1.11.31 PM

You can find additional examples and documentation on the dygraphs for R website.

Bringing JavaScript to R

One of the reasons we are excited about dygraphs is that it takes a mature and feature rich visualization library formerly only accessible to web developers and makes it available to all R users.

This is part of a larger trend enabled by the htmlwidgets package, and we expect that more and more libraries like dygraphs will emerge over the coming months to bring the best of JavaScript data visualization to R.

 

Over the past several years the Rcpp package has become an indispensable tool for creating high-performance R code. Its power and ease of use have made C++ a natural second language for many R users. There are over 400 packages on CRAN and Bioconductor that depend on Rcpp and it is now the most downloaded R package.

In RStudio v0.99 we have added extensive additional tools to make working with Rcpp more pleasant, productive, and robust, these include:

  • Code completion
  • Source diagnostics as you edit
  • Code snippets
  • Auto-indentation
  • Navigable list of compilation errors
  • Code navigation (go to definition)

We think these features will go a long way to helping even more R users succeed with Rcpp. You can try the new features out now by downloading the RStudio Preview Release.

Code Completion

RStudio v0.99 includes comprehensive code completion for C++ based on Clang (the same underlying engine used by XCode and many other C/C++ tools):

Screen Shot 2015-04-07 at 12.13.31 PM

Completions are provided for the C++ language, Rcpp, and any other libraries you have imported.

Diagnostics

As you edit C++ source files RStudio uses Clang to scan your code looking for errors, incomplete code, or other conditions worthy of warnings or informational notes. For example:

Screen Shot 2015-04-07 at 12.16.38 PM

Diagnostics alert you to the possibility of subtle problems and flag outright incorrect code as early as possible, substantially reducing iteration/debugging time.

Interactive C++

Rcpp includes some nifty tools to help make working with C++ code just as simple and straightforward as working with R code. You can “source” C++ code into R just like you’d source an R script (no need to deal with Makefiles or build systems). Here’s a Gibbs Sampler implemented with Rcpp:

Screen Shot 2015-04-13 at 4.40.36 PM

We can make this function available to R by simply sourcing the C++ file (much like we’d source an R script):

sourceCpp("gibbs.cpp")
gibbs(100, 10)

Thanks to the abstractions provided by Rcpp, the code implementing the Gibbs Sampler in C++ is nearly identical to the code you’d write in R, but runs 20 times faster. RStudio includes full support for Rcpp’s sourceCpp via the Source button and Ctrl+Shift+Enter keyboard shortcut.

Try it Out

If you are new to C++ or Rcpp you might be surprised at how easy it is to get started. There are lots of great resources available, including:

You can give the new Rcpp features a try now by downloading the RStudio Preview Release. If you run into problems or have feedback on how we could make things better let us know on our Support Forum.

We’re getting close to shipping the next version of RStudio (v0.99) and this week will continue our series of posts describing the major new features of the release (previous posts have already covered code completion, the revamped data viewer, and improvements to vim mode). Note that if you want to try out any of the new features now you can do so by downloading the RStudio Preview Release.

Code Snippets

Code snippets are text macros that are used for quickly inserting common snippets of code. For example, the fun snippet inserts an R function definition:

Insert Snippet

If you select the snippet from the completion list it will be inserted along with several text placeholders which you can fill in by typing and then pressing Tab to advance to the next placeholder:

Screen Shot 2015-04-07 at 10.44.39 AM

Other useful snippets include:

  • lib, req, and source for the library, require, and source functions
  • df and mat for defining data frames and matrices
  • if, el, and ei for conditional expressions
  • apply, lapply, sapply, etc. for the apply family of functions
  • sc, sm, and sg for defining S4 classes/methods.

Snippets are a great way to automate inserting common/boilerplate code and are available for R, C/C++, JavaScript, and several other languages.

Inserting Snippets

As illustrated above, code snippets show up alongside other code completion results and can be inserted by picking them from the completion list. By default the completion list will show up automatically when you pause typing for 250 milliseconds and can also be manually activated via the Tab key. In addition, if you have typed the character sequence for a snippet and want to insert it immediately (without going through the completion list) you can press Shift+Tab.

Customizing Snippets

You can edit the built-in snippet definitions and even add snippets of your own via the Edit Snippets button in Global Options -> Code:

Edit Snippets

Custom snippets are defined using the snippet keyword. The contents of the snippet should be indented below using the <tab> key (rather than with spaces). Variables can be defined using the form {1:varname}. For example, here’s the definition of the setGeneric snippet:

snippet sg
  setGeneric("${1:generic}", function(${2:x, ...}) {
    standardGeneric("${1:generic}")
  })

Once you’ve customized snippets for a given language they are written into the ~/.R/snippets directory. For example, the customized versions of R and C/C++ snippets are written to:

~/.R/snippets/r.snippets
~/.R/snippets/c_cpp.snippets

You can edit these files directly to customize snippet definitions or you can use the Edit Snippets dialog as described above. If you need to move custom snippet definitions to another system then simply place them in ~/.R/snippets and they’ll be used in preference to the built-in snippet definitions.

Try it Out

You can give code snippets a try now by downloading the RStudio Preview Release. If you run into problems or have feedback on how we could make things better let us know on our Support Forum.

We’re busy at work on the next version of RStudio (v0.99) and this week will be blogging about some of the noteworthy new features. If you want to try out any of the new features now you can do so by downloading the RStudio Preview Release.

The first feature to highlight is a fully revamped implementation of code completion for R. We’ve always supported a limited form of completion however (a) it only worked on objects in the global environment; and (b) it only worked when expressly requested via the tab key. As a result not nearly enough users discovered or benefitted from code completion. In this release code completion is much more comprehensive.

Smarter Completion Engine

Previously RStudio only completed variables that already existed in the global environment, now completion is done based on source code analysis so is provided even for objects that haven’t been fully evaluated:

document-inferred

Completions are also provided for a wide variety of specialized contexts including dimension names in [ and [[:

bracket

RStudio now provides completions for function arguments within function chains using magrittr’s %>% operator, for e.g. dplyr data transformation pipelines. Extending this behavior, we also provide the appropriate completions for the various ‘verbs’ used by dplyr:

dplyr        dplyr_verb

In addition, certain functions, such as library() and require(), expect package names for completions. RStudio automatically infers whether a particular function expects a package name and provides those names as completion results:

library

Completion is now also S3 and S4 aware. If RStudio is able to determine which method a particular function call will be dispatched to it will attempt to retrieve completions from that method. For example, the sort.default() method provides an extra argument, na.last, not available in the sort() generic. RStudio will provide completions for that argument if S3 dispatch would choose sort.default()

s3

Beyond what’s described above there are lots more new places where completions are provided:

  • For Shiny applications, completions for ui.R + server.R pairs
  • Completions for knitr options, e.g. in opts_chunk$get(), are now supplied
  • Completions for dynamic symbols within .C, .Call, .Fortran, .External

Additional Enhancements

Always On Completion

Previously RStudio only displayed completions “on-demand” in response to the tab key. Now, RStudio will proactively display completions after a $ or :: as well as after a period of typing inactivity. All of this behavior is configurable via the new completion options panel:

options

File Completions

When within an RStudio project, completions will be applied recursively to all file names matching the current token. The enclosing parent directory is printed on the right:

file

Fuzzy Narrowing

Got a completion with an excessively long name, perhaps a particularly long named Bioconductor package, or another variable or function name of long length? RStudio now uses ‘fuzzy narrowing’ on the completion list, by checking to see if the completion matches a ‘subsequence’ within each completion. By subsequence, we mean a sequence of characters not necessarily connected within the completion, so that for example, ‘fpse’ could match ‘file_path_sans_extension’. We hope that users will quickly become accustomed to this behavior and find it very useful.

fuzzy

Trying it Out

We think that the new completion features make for a qualitatively better experience of writing R code for beginning and expert users alike.  You can give the new features a try now by downloading the RStudio Preview Release.  If you run into problems or have feedback on how we could make things better let us know on our Support Forum.

 

Today we’re excited to announce htmlwidgets, a new framework that brings the best of JavaScript data visualization libraries to R. There are already several packages that take advantage of the framework (leaflet, dygraphs, networkD3, DataTables, and rthreejs) with hopefully many more to come.

An htmlwidget works just like an R plot except it produces an interactive web visualization. A line or two of R code is all it takes to produce a D3 graphic or Leaflet map. Widgets can be used at the R console as well as embedded in R Markdown reports and Shiny web applications. Here’s an example of using leaflet directly from the R console:

rconsole.2x

When printed at the console the leaflet widget displays in the RStudio Viewer pane. All of the tools typically available for plots are also available for widgets, including history, zooming, and export to file/clipboard (note that when not running within RStudio widgets will display in an external web browser).

Here’s the same widget in an R Markdown report. Widgets automatically print as HTML within R Markdown documents and even respect the default knitr figure width and height.

rmarkdown.2x

Widgets also provide Shiny output bindings so can be easily used within web applications. Here’s the same widget in a Shiny application:

shiny.2x

Bringing JavaScript to R

The htmlwidgets framework is a collaboration between Ramnath Vaidyanathan (rCharts), Kenton Russell (Timely Portfolio), and RStudio. We’ve all spent countless hours creating bindings between R and the web and were motivated to create a framework that made this as easy as possible for all R developers.

There are a plethora of libraries available that create attractive and fully interactive data visualizations for the web. However, the programming interface to these libraries is JavaScript, which places them outside the reach of nearly all statisticians and analysts. htmlwidgets makes it extremely straightforward to create an R interface for any JavaScript library.

Here are a few widget libraries that have been built so far:

  • leaflet, a library for creating dynamic maps that support panning and zooming, with various annotations like markers, polygons, and popups.
  • dygraphs, which provides rich facilities for charting time-series data and includes support for many interactive features including series/point highlighting, zooming, and panning.
  • networkD3, a library for creating D3 network graphs including force directed networks, Sankey diagrams, and Reingold-Tilford tree networks.
  • DataTables, which displays R matrices or data frames as interactive HTML tables that support filtering, pagination, and sorting.
  • rthreejs, which features 3D scatterplots and globes based on WebGL.

All of these libraries combine visualization with direct interactivity, enabling users to explore data dynamically. For example, time-series visualizations created with dygraphs allow dynamic panning and zooming:

NewHavenTemps

Learning More

To learn more about the framework and see a showcase of the available widgets in action check out the htmlwidgets web site. To learn more about building your own widgets, install the htmlwidgets package from CRAN and check out the developer documentation.

 

Today we’re very pleased to announce a new version of RStudio (v0.98.932) which is available for download now. New features in this release include:

  • A next generation implementation of R Markdown with a raft of new features including support for HTML, PDF, and Word output, many new options for customizing document appearance, and the ability to create presentations (Beamer or HTML5).
  • Interactive Documents (Shiny meets R Markdown). Readers can now change the parameters underlying your analysis and see the results immediately. Interactive Documents make it easier than ever to use Shiny!
  • Shareable notebooks from R scripts. Notebooks include all R code and generated output, and can be rendered in HTML, PDF, and Word formats.
  • Enhanced debugging including support for the new R 3.1 debugging commands to step into function calls and finish the current loop or function.
  • Various source editor enhancements including new syntax highlighting modes for XML, YAML, SQL, Python, and shell scripts. You can also execute Python and shell scripts directly from the editor using Ctrl+Shift+Enter.
  • Integrated tools for Shiny development including the ability to run applications within an IDE pane as well as Run/Reload applications with a keyboard shortcut (Ctrl+Shift+Enter).
  • A new devtools mode for package development (uses devtools for check, document, test, build, etc.)
  • Contextual Git/SVN menu that enables quick access to per-file revision history and selection-aware View/Blame for projects hosted on GitHub.
  • Fast lookup of shortcuts using the new keyboard shortcut quick-reference card (Alt+Shift+K)

See the release notes for a full list of what’s changed and see Yihui Xie’s post on R Markdown v2 for more on what’s new there.

We’ll be posting additional articles over the next few days that describe the new features in more depth. In the meantime we hope you download the new version and as always let us know how it’s working and what else you’d like to see.

Today, we’re excited to announce the release of Shiny Server version 0.4 as well as the availability of a beta version of Shiny Server Professional Edition.

Shiny Server is a platform for hosting Shiny Applications over the Web and has undergone substantial work in the past few months. We have fixed many bugs, added stability enhancements, and have created pre-built installers for Ubuntu 12.04 (and later) and RedHat/CentOS 5 and 6. The new installers will drastically simplify the process of installing and configuring Shiny Server on these distributions. For other platforms you can use the updated instructions to build from source.

Important note for current Shiny Server users: We are no longer relying on npm to distribute the software. If you had previously installed version 0.3.x using npm, you must uninstall the old version before upgrading. Follow these instructions to uninstall the old version before upgrading to the new.

We hope this new version will allow you to deploy your Shiny applications even more efficiently. Please reach out on the mailing list to let us know what you think or if you have any problems.

Shiny Server Pro beta

We’ve recently begun beta testing of Shiny Server Professional Edition. This product adds features that make it easier for an enterprise to scale, tune, monitor and receive support for production environments.  Shiny Server Pro will include the ability to configure a Shiny application with more than one process, and control the number of concurrent users per application. It adds an administrative dashboard to monitor and gain insight into your applications, and includes integrations with a variety of authentication systems including LDAP and Active Directory.

If you’re interested in finding out more about Shiny Server Pro, or being a participant in our beta please register here.

Thank you for your help in making Shiny Server a better product.  We hope you enjoy Shiny Server 0.4 and look forward to getting your feedback.

We’re pleased to announce that the final version of RStudio v0.98 is available for download now. Highlights of the new release include:

  • An interactive debugger for R that is tightly integrated with base R debugging tools (browser, recover, etc.)
  • Numerous improvements to the Workspace pane (which is now called the Environment pane).
  • R Presentations for easy authoring of HTML5 presentations that include R code, output, and graphics.
  • A new Viewer pane for displaying local web content (e.g. graphical output from packages like googleVis).
  • Additional support for developing and running Shiny web applications.
  • Substantially improved UI performance on Mac OS X.
  • Professional Edition of RStudio Server with many new capabilities for enterprise deployment.

There are also lots of smaller improvements and bug fixes across the product, check out the release notes for full details.

Debugging Tools

The feature we’re most excited about is the addition of a full interactive debugger to the IDE. Noteworthy capabilities of the debugger include:

  • Setting breakpoints within the source editor, both inside and outside functions
  • Stepping through code line by line
  • Inspecting object values and the call stack during debugging
  • An error inspector for quick access to tracebacks and the debugger after runtime errors
  • Tight integration with traditional R debugging tools, such as browser() and debug()

Here’s a screenshot of the IDE after hitting an editor breakpoint:

RStudioDebugger

For more details on how to take advantage of the new debugging tools, see Debugging with RStudio.

Environment Pane

The Workspace pane is now called the Environment pane and has numerous improvements, including:

  • Browse any environment on the search path
  • Filtering by name/value
  • Expand lists, data frames, and S4 objects inline
  • Use str() to display object values
  • Optional grid view sortable by various attributes
  • Many other small correctness and robustness enhancements

R Presentations

R Presentations enable easy authoring of HTML5 presentations. R Presentations are based on R Markdown, and include the following features:

  • Easy authoring of HTML5 presentations based on R Markdown
  • Extensive support for authoring and previewing inside the IDE
  • Many options for customizing layout and appearance
  • Publishing as either a standalone HTML file or to RPubs

Here’s a screenshot showing a simple presentation being authored and previewed within the IDE:

RPresentations

For more details see the documentation on Authoring R Presentations.

Viewer Pane

RStudio now includes a Viewer pane that can be used to view local web content. This includes both static web content or even a local web application created using ShinyRook, or OpenCPU. This is especially useful for packages that have R bindings to Javascript data visualization libraries.

The googleVis and rCharts packages have already been updated to take advantage of the Viewer pane. Here’s a screenshot of the googleVis integration:

googleVis

We’re hopeful that there will be many more compelling uses of the Viewer. For more details see the article Extending RStudio with the Viewer Pane.

Shiny Integration

We’ve added a number of features to support development of Shiny web applications, including:

  • The ability to develop and run Shiny applications on RStudio Server (localhost and websocket proxying is handled automatically)
  • Running Shiny applications within an IDE pane (see the discussion of the Viewer pane below for details)
  • Create a new Shiny application from within the New Project dialog
  • Debugging of Shiny applications using the new RStudio debugging tools.

Mac UI Framework

In RStudio v0.98 we also migrated our Mac WebKit engine from a cross-platform framework (Qt) to Cocoa. The original motivation for this was compatibility problems between Qt and OS X Mavericks, but as it turned out the move to Cocoa WebKit yielded substantially faster editor, scrolling, layout, and graphics performance across the board. If you are a Mac user you’ll find everything about the product snappier in v0.98.

In the next major version of RStudio we’re hoping to make comparable improvements in performance on both Linux and Windows by using a more modern WebKit on those platforms as well.

RStudio Server Professional Edition

Over the years we’ve gotten lots of feedback from larger organizations deploying RStudio Server on the features they’d like to see for production deployments of the server. With RStudio v0.98 we’re introducing a new Professional Edition of RStudio Server that incorporates much of this feedback. Highlights include:

  • An administrative dashboard that provides insight into active sessions, server health, and monitoring of system-wide and per-user performance and resource metrics.
  • Authentication using system accounts, ActiveDirectory, LDAP, or Google Accounts.
  • Full support for PAM (including PAM sessions for dynamically provisioning user resources).
  • Ability to establish per-user or per-group CPU priorities and memory limits.
  • HTTP enhancements including support for SSL and keep-alive for improved performance.
  • Ability to restrict access to the server by IP.
  • Customizable server health checks.
  • Suspend, terminate, or assume control of user sessions.
  • Impersonate users for assistance and troubleshooting.

The RStudio Server product page has full details on the Professional Edition, and an evaluation version of the server is also available for download.

New Support Site

With this release we’re also introducing a brand new support and documentation website, please visit us there with questions, feedback, as well as what other improvements you’d like to see in the product.

When OS X Mavericks was released last month we were very disappointed to discover a compatibility issue between Qt (our cross-platform user interface toolkit) and OS X Mavericks that resulted in extremely poor graphics performance.

We now have an updated preview version of RStudio for OS X (v0.98.475) that not only overcomes these issues, but also improves editor, scrolling, and layout performance across the board on OS X (more details below if you are curious):

http://www.rstudio.com/ide/download/preview

We were initially optimistic that we could patch Qt to overcome the problems but even with some help from Digia (the organization behind Qt) we never got acceptable performance. Running out of viable options based on Qt, we decided to bypass Qt entirely by implementing the RStudio desktop frame as a native Cocoa application.

OS X Mavericks issues aside, we are thrilled with the result of using Cocoa rather than a cross-platform toolkit. RStudio desktop uses WebKit to render its user-interface, and the Cocoa WebKit Framework is substantially faster than the one in Qt.

Please try out the updated preview and let us know if you encounter any issues or problems on our support forum. For those that prefer to wait for the final release of v0.98 we expect that to happen sometime during the next couple of weeks.

UPDATERStudio OS X Mavericks Issues Resolved

This post is now out of date (see link above for information on getting a version of RStudio that works with OS X Mavericks).


Today Apple released OS X 10.9 “Mavericks”. If you are a Mac user and considering updating to the new OS  there are some RStudio compatibility issues to consider before you update.

As a result of a problem between Mavericks and the user interface toolkit underlying RStudio (Qt) the RStudio IDE is very slow in painting and user interactions  when running under Mavericks. We are following up with both Qt and Apple on resolving the compatibility issue. In the meantime there is a workaround available in the v0.98.443 release of RStudio that can be downloaded here:

http://www.rstudio.com/ide/download/preview

This version of RStudio detects when it is running on OS X Mavericks and in that case bypasses the use of Qt. Rather, a version of RStudio Server is run locally and connected to by a special RStudioIDE browser window. There are several differences you’ll notice when running in this mode:

  1. Only one instance of RStudio can be run at a time.
  2. The Mac native menubar is not used. Rather, the main menu appears inside the RStudio frame.
  3. Mac native file open and save dialogs are not used. Rather, internal versions of the dialogs are used.
  4. Finder file associations activate RStudio however don’t open the targeted file(s).
  5. The copy plot to clipboard function is not available.
  6. During a shutdown of Mac OS X when RStudio is running the current project’s Workspace is not saved automatically (however source files are).

We’re hoping that the underlying problem in OS X 10.9 is resolved in a future update or alternatively the Qt toolkit is updated to address the issue. If and when that occurs we’ll release a new version of RStudio that restores the previous RStudio behavior on OS X 10.9.

Follow

Get every new post delivered to your Inbox.

Join 12,627 other followers