You are currently browsing jjallaire’s articles.

Traditionally, the mechanisms for obtaining R and related software have used standard HTTP connections. This isn’t ideal though, as without a secure (HTTPS) connection there is less assurance that you are downloading code from a legitimate source rather than from another server posing as one.

Recently there have been a number of changes that make it easier to use HTTPS for installing R, RStudio, and packages from CRAN:

  1. Downloads of R from the main CRAN website now use HTTPS;

  2. Downloads of RStudio from our website now use HTTPS; and

  3. It is now possible to install packages from CRAN over HTTPS.

There are a number of ways to ensure that installation of packages from CRAN are performed using HTTPS. The most recent version of R  (v3.2.2) makes this the default behavior. The most recent version of RStudio (v0.99.473) also attempts to configure secure downloads from CRAN by default (even for older versions of R). Finally, any version of R or RStudio can use secure HTTPS downloads by making some configuration changes as described in the Secure Package Downloads for R article in our Knowledge Base.

Configuring Secure Connections to CRAN

While the simplest way to ensure secure connections to CRAN is to run the updated versions mentioned above, it’s important to note that it is not necessary to upgrade R or RStudio to achieve this end. Rather, two configuration changes can be made:

  1. The R download.file.method option needs to specify a method that is capable of HTTPS; and
  2. The CRAN mirror you are using must be capable of HTTPS connections (not all of them are).

The specifics of the required changes for various products, platforms, and versions of R are described in-depth in the Secure Package Downloads for R article in our Knowledge Base.

Recommendations for RStudio Users

We’ve made several changes to RStudio IDE to ensure that HTTPS connections are used throughout the product:

  1. The default download.file.method option is set to an HTTPS compatible method (with a warning displayed if a secure method can’t be set);
  2. The configured CRAN mirror is tested for HTTPS compatibility and a warning is displayed if the mirror doesn’t support HTTPS;
  3. HTTPS is used for user selection of a non-default CRAN mirror;
  4. HTTPS is used for in-product documentation links;
  5. HTTPS is used when checking for updated versions of RStudio (applies to desktop version only); and
  6. HTTPS is used when downloading Rtools (applies to desktop version only).

If you are running RStudio on the desktop we strongly recommend that you update to the latest version (v0.99.473).

Recommendations for Server Administrators

If you are running RStudio Server it’s possible to make the most important security enhancements by changing your configuration rather than updating to a new version. The Secure Package Downloads for R article in our Knowledge Base provides documentation on how do this.

In this case in-product documentation links and user selection of a non-default CRAN mirror will continue to use HTTP rather than HTTPS however these are less pressing concerns than CRAN package installation. If you’d like these functions to also be performed over HTTPS then you should upgrade your server to the latest version of RStudio.

If you are running Shiny Server we recommend that you modify your configuration to support HTTPS package downloads as described in the Secure Package Downloads for R article.

To paraphrase Yogi Berra, “Predicting is hard, especially about the future”. In 1993, when Ross Ihaka and Robert Gentleman first started working on R, who would have predicted that it would be used by millions in a world that increasingly rewards data literacy? It’s impossible to know where R will go in the next 20 years, but at RStudio we’re working hard to make sure the future is bright.

Today, we’re excited to announce our participation in the R Consortium, a new 501(c)6 nonprofit organization. The R Consortium is a collaboration between the R Foundation, RStudio, Microsoft, TIBCO, Google, Oracle, HP and others. It’s chartered to fund and inspire ideas that will enable R to become an even better platform for science, research, and industry. The R Consortium complements the R Foundation by providing a convenient funding vehicle for the many commercial beneficiaries of R to give back to the community, and will provide the resources to embark on ambitious new projects to make R even better.

We believe the R Consortium is critically important to the future of R and despite our small size, we chose to join it at the highest contributor level (alongside Microsoft). Open source is a key component of our mission and giving back to the community is extremely important to us.

The community of R users and developers have a big stake in the language and its long-term success. We all want free and open source R to continue thriving and growing for the next 20 years and beyond. The fact that so many of the technology industry’s largest companies are willing to stand behind R as part of the consortium is remarkable and we think bodes incredibly well for the future of R.

We’re pleased to announce that the final version of RStudio v0.99 is available for download now. Highlights of the release include:

  • A new data viewer with support for large datasets, filtering, searching, and sorting.
  • Complete overhaul of R code completion with many new features and capabilities.
  • The source editor now provides code diagnostics (errors, warnings, etc.) as you work.
  • User customizable code snippets for automating common editing tasks.
  • Tools for Rcpp: completion, diagnostics, code navigation, find usages, and automatic indentation.
  • Many additional source editor improvements including multiple cursors, tab re-ordering, and several new themes.
  • An enhanced Vim mode with visual block selection, macros, marks, and subset of : commands.

There are also lots of smaller improvements and bug fixes across the product. Check out the v0.99 release notes for details on all of the changes.

Data Viewer

We’ve completely overhauled the data viewer with many new capabilities including live update, sorting and filtering, full text searching, and no row limit on viewed datasets.

data-viewer

See the data viewer documentation for more details.

Code Completion

Previously RStudio only completed variables that already existed in the global environment. Now completion is done based on source code analysis so is provided even for objects that haven’t been fully evaluated:

completion-scopes

Completions are also provided for a wide variety of specialized contexts including dimension names in [ and [[:

completion-bracket

Code Diagnostics

We’ve added a new inline code diagnostics feature that highlights various issues in your R code as you edit.

For example, here we’re getting a diagnostic that notes that there is an extra parentheses:

Screen Shot 2015-04-08 at 12.04.14 PM

Here the diagnostic indicates that we’ve forgotten a comma within a shiny UI definition:

diagnostics-comma

A wide variety of diagnostics are supported, including optional diagnostics for code style issues (e.g. the inclusion of unnecessary whitespace). Diagnostics are also available for several other languages including C/C++, JavaScript, HTML, and CSS. See the code diagnostics documentation for additional details.

Code Snippets

Code snippets are text macros that are used for quickly inserting common snippets of code. For example, the fun snippet inserts an R function definition:

Insert Snippet

If you select the snippet from the completion list it will be inserted along with several text placeholders which you can fill in by typing and then pressing Tab to advance to the next placeholder:

Screen Shot 2015-04-07 at 10.44.39 AM

Other useful snippets include:

  • lib, req, and source for the library, require, and source functions
  • df and mat for defining data frames and matrices
  • if, el, and ei for conditional expressions
  • apply, lapply, sapply, etc. for the apply family of functions
  • sc, sm, and sg for defining S4 classes/methods.

See the code snippets documentation for additional details.

Try it Out

RStudio v0.99 is available for download now. We hope you enjoy the new release and as always please let us know how it’s working and what else we can do to make the product better.

Soon after the announcement of htmlwidgets, Rich Iannone released the DiagrammeR package, which makes it easy to generate graph and flowchart diagrams using text in a Markdown-like syntax. The package is very flexible and powerful, and includes:

  1. Rendering of Graphviz graph visualizations (via viz.js)
  2. Creating diagrams and flowcharts using mermaid.js
  3. Facilities for mapping R objects into graphs, diagrams, and flowcharts.

We’re very excited about the prospect of creating sophisticated diagrams using an easy to author plain-text syntax, and built some special authoring support for DiagrammeR into RStudio v0.99 (which you can download a preview release of now).

Graphviz Meets R

If you aren’t familiar with Graphviz, it’s a tool for rendering DOT (a plain text graph description language). DOT draws directed graphs as hierarchies. Its features include well-tuned layout algorithms for placing nodes and edge splines, edge labels, “record” shapes with “ports” for drawing data structures, and cluster layouts (see http://www.graphviz.org/pdf/dotguide.pdf for an introductory guide).

DiagrammeR can render any DOT script. For example, with the following source file (“boxes.dot”):

Screen Shot 2015-04-30 at 12.35.17 PM

You can render the diagram with:

library(DiagrammeR)
grViz("boxes.dot")

grviz-viewer

Since the diagram is an htmlwidget it can be used at the R console, within R Markdown documents, and within Shiny applications. Within RStudio you can preview a Graphviz or mermaid source file the same way you source an R script via the Preview button or the Ctrl+Shift+Enter keyboard shortcut.

This simple example only scratches the surface of what’s possible, see the DiagrammeR Graphviz documentation for more details and examples.

Diagrams with mermaid.js

Support for mermaid.js in DiagrammeR enables you to create several other diagram types not supported by Graphviz. For example, here’s the code required to create a sequence diagram:

sequence

You can render the diagram with:

library(DiagrammeR)
mermaid("sequence.mmd")

sequence-viewer

See the DigrammeR mermaid.js documentation for additional details.

Generating Diagrams from R Code

Both of the examples above illustrating creating diagrams by direct editing of DOT and mermaid scripts. The latest version of DiagrammeR (v0.6, just released to CRAN) also includes facilities for generating diagrams from R code. This can be done in a couple of ways:

  1. Using text substitution, whereby you create placeholders within the diagram script and substitute their values from R objects. See the documentation on Graphviz Substitution for more details.
  2. Using the graphviz_graph function you can specify nodes and edges directly using a data frame.

Future versions of DiagrammeR are expected to include additional features to support direct generation of diagrams from R.

Publishing with DiagrammeR

Diagrams created with DiagrammeR act a lot like R plots however there’s an important difference: they are rendered as HTML content rather than using an R graphics device. This has the following implications for how they can be published and re-used:

  1. Within RStudio you can save diagrams as an image (PNG, BMP, etc.) or copy them to clipboard for re-use in other applications.
  2. For a more reproducible workflow, diagrams can be embedded within R Markdown documents just like plots (all of the required HTML and JS is automatically included). Note that because the diagrams depend on HTML and JavaScript for rendering they can only be used in HTML based output formats (they don’t work in PDFs or MS Word documents).
  3. From within RStudio you can also publish diagrams to RPubs or save them as standalone web pages.

diagrammer-publish

See the DiagrammeR documentation on I/O for additional details.

Try it Out

To get started with DiagrammeR check out the excellent collection of demos and documentation on the project website. To take advantage of the new RStudio features that support DiagrammeR you should download the latest RStudio v0.99 Preview Release.

 

 

 

In RStudio v0.99 we’ve made a major investment in R source code analysis. This work resulted in significant improvements in code completion, and in the latest preview release enable a new inline code diagnostics feature that highlights various issues in your R code as you edit.

For example, here we’re getting a diagnostic that notes that there is an extra parentheses:

Screen Shot 2015-04-08 at 12.04.14 PM

Here the diagnostic indicates that we’ve forgotten a comma within a shiny UI definition:

diagnostics-comma

This diagnostic flags an unknown parameter to a function call:

Screen Shot 2015-04-08 at 11.50.07 AM

This diagnostic indicates that we’ve referenced a variable that doesn’t exist and suggests a fix based on another variable in scope:

Screen Shot 2015-04-08 at 4.23.49 PM

A wide variety of diagnostics are supported, including optional diagnostics for code style issues (e.g. the inclusion of unnecessary whitespace). Diagnostics are also available for several other languages including C/C++, JavaScript, HTML, and CSS.

Configuring Diagnostics

By default, code in the current source file is checked whenever it is saved, as well as if the keyboard is idle for a period of time. You can tweak this behavior using the Code -> Diagnostics options:

diagnostics-options

Note that several of the available diagnostics are disabled by default. This is because we’re in the process of refining their behavior to eliminate “false negatives” where correct code is flagged as having a problem. We’ll continue to improve these diagnostics and enable them by default when we feel they are ready.

Trying it Out

You can try out the new code diagnostics by downloading the latest preview release of RStudio. This feature is a work in progress and we’re particularly interested in feedback on how well it works. Please also let us know if there are common coding problems which you think we should add new diagnostics for. We hope you try out the preview and let us know how we can make it better.

 

The dygraphs package is an R interface to the dygraphs JavaScript charting library. It provides rich facilities for charting time-series data in R, including:

  • Automatically plots xts time-series objects (or objects convertible to xts).
  • Rich interactive features including zoom/pan and series/point highlighting.
  • Highly configurable axis and series display (including optional 2nd Y-axis).
  • Display upper/lower bars (e.g. prediction intervals) around series.
  • Various graph overlays including shaded regions, event lines, and annotations.
  • Use at the R console just like conventional R plots (via RStudio Viewer).
  • Embeddable within R Markdown documents and Shiny web applications.

The dygraphs package is available on CRAN now and can be installed with:

install.packages("dygraphs")

Examples

Here are some examples of interactive time series visualizations you can create with only a line or two of R code (the screenshots are static, click them to see the interactive version).

Panning and Zooming

This code adds a range selector that’s can be used to pan and zoom around the series data:

dygraph(nhtemp, main = "New Haven Temperatures") %>%
  dyRangeSelector()

Screen Shot 2015-04-09 at 1.01.35 PM

Point Highlighting

When you hover over the time-series the values of all points at the location of the mouse are shown in the legend:

lungDeaths <- cbind(ldeaths, mdeaths, fdeaths)
dygraph(lungDeaths, main = "Deaths from Lung Disease (UK)") %>%
  dyOptions(colors = RColorBrewer::brewer.pal(3, "Set2"))

Screen Shot 2015-04-09 at 12.53.54 PM

Shading and Annotations

There are a wide variety of tools available to annotate time series. Here we demonstrate creating shaded regions:

dygraph(nhtemp, main="New Haven Temperatures") %>% 
  dySeries(label="Temp (F)", color="black") %>%
  dyShading(from="1920-1-1", to="1930-1-1", color="#FFE6E6") %>%
  dyShading(from="1940-1-1", to="1950-1-1", color="#CCEBD6")

Screen Shot 2015-04-09 at 1.11.31 PM

You can find additional examples and documentation on the dygraphs for R website.

Bringing JavaScript to R

One of the reasons we are excited about dygraphs is that it takes a mature and feature rich visualization library formerly only accessible to web developers and makes it available to all R users.

This is part of a larger trend enabled by the htmlwidgets package, and we expect that more and more libraries like dygraphs will emerge over the coming months to bring the best of JavaScript data visualization to R.

 

Over the past several years the Rcpp package has become an indispensable tool for creating high-performance R code. Its power and ease of use have made C++ a natural second language for many R users. There are over 400 packages on CRAN and Bioconductor that depend on Rcpp and it is now the most downloaded R package.

In RStudio v0.99 we have added extensive additional tools to make working with Rcpp more pleasant, productive, and robust, these include:

  • Code completion
  • Source diagnostics as you edit
  • Code snippets
  • Auto-indentation
  • Navigable list of compilation errors
  • Code navigation (go to definition)

We think these features will go a long way to helping even more R users succeed with Rcpp. You can try the new features out now by downloading the RStudio Preview Release.

Code Completion

RStudio v0.99 includes comprehensive code completion for C++ based on Clang (the same underlying engine used by XCode and many other C/C++ tools):

Screen Shot 2015-04-07 at 12.13.31 PM

Completions are provided for the C++ language, Rcpp, and any other libraries you have imported.

Diagnostics

As you edit C++ source files RStudio uses Clang to scan your code looking for errors, incomplete code, or other conditions worthy of warnings or informational notes. For example:

Screen Shot 2015-04-07 at 12.16.38 PM

Diagnostics alert you to the possibility of subtle problems and flag outright incorrect code as early as possible, substantially reducing iteration/debugging time.

Interactive C++

Rcpp includes some nifty tools to help make working with C++ code just as simple and straightforward as working with R code. You can “source” C++ code into R just like you’d source an R script (no need to deal with Makefiles or build systems). Here’s a Gibbs Sampler implemented with Rcpp:

Screen Shot 2015-04-13 at 4.40.36 PM

We can make this function available to R by simply sourcing the C++ file (much like we’d source an R script):

sourceCpp("gibbs.cpp")
gibbs(100, 10)

Thanks to the abstractions provided by Rcpp, the code implementing the Gibbs Sampler in C++ is nearly identical to the code you’d write in R, but runs 20 times faster. RStudio includes full support for Rcpp’s sourceCpp via the Source button and Ctrl+Shift+Enter keyboard shortcut.

Try it Out

If you are new to C++ or Rcpp you might be surprised at how easy it is to get started. There are lots of great resources available, including:

You can give the new Rcpp features a try now by downloading the RStudio Preview Release. If you run into problems or have feedback on how we could make things better let us know on our Support Forum.

We’re getting close to shipping the next version of RStudio (v0.99) and this week will continue our series of posts describing the major new features of the release (previous posts have already covered code completion, the revamped data viewer, and improvements to vim mode). Note that if you want to try out any of the new features now you can do so by downloading the RStudio Preview Release.

Code Snippets

Code snippets are text macros that are used for quickly inserting common snippets of code. For example, the fun snippet inserts an R function definition:

Insert Snippet

If you select the snippet from the completion list it will be inserted along with several text placeholders which you can fill in by typing and then pressing Tab to advance to the next placeholder:

Screen Shot 2015-04-07 at 10.44.39 AM

Other useful snippets include:

  • lib, req, and source for the library, require, and source functions
  • df and mat for defining data frames and matrices
  • if, el, and ei for conditional expressions
  • apply, lapply, sapply, etc. for the apply family of functions
  • sc, sm, and sg for defining S4 classes/methods.

Snippets are a great way to automate inserting common/boilerplate code and are available for R, C/C++, JavaScript, and several other languages.

Inserting Snippets

As illustrated above, code snippets show up alongside other code completion results and can be inserted by picking them from the completion list. By default the completion list will show up automatically when you pause typing for 250 milliseconds and can also be manually activated via the Tab key. In addition, if you have typed the character sequence for a snippet and want to insert it immediately (without going through the completion list) you can press Shift+Tab.

Customizing Snippets

You can edit the built-in snippet definitions and even add snippets of your own via the Edit Snippets button in Global Options -> Code:

Edit Snippets

Custom snippets are defined using the snippet keyword. The contents of the snippet should be indented below using the <tab> key (rather than with spaces). Variables can be defined using the form {1:varname}. For example, here’s the definition of the setGeneric snippet:

snippet sg
  setGeneric("${1:generic}", function(${2:x, ...}) {
    standardGeneric("${1:generic}")
  })

Once you’ve customized snippets for a given language they are written into the ~/.R/snippets directory. For example, the customized versions of R and C/C++ snippets are written to:

~/.R/snippets/r.snippets
~/.R/snippets/c_cpp.snippets

You can edit these files directly to customize snippet definitions or you can use the Edit Snippets dialog as described above. If you need to move custom snippet definitions to another system then simply place them in ~/.R/snippets and they’ll be used in preference to the built-in snippet definitions.

Try it Out

You can give code snippets a try now by downloading the RStudio Preview Release. If you run into problems or have feedback on how we could make things better let us know on our Support Forum.

We’re busy at work on the next version of RStudio (v0.99) and this week will be blogging about some of the noteworthy new features. If you want to try out any of the new features now you can do so by downloading the RStudio Preview Release.

The first feature to highlight is a fully revamped implementation of code completion for R. We’ve always supported a limited form of completion however (a) it only worked on objects in the global environment; and (b) it only worked when expressly requested via the tab key. As a result not nearly enough users discovered or benefitted from code completion. In this release code completion is much more comprehensive.

Smarter Completion Engine

Previously RStudio only completed variables that already existed in the global environment, now completion is done based on source code analysis so is provided even for objects that haven’t been fully evaluated:

document-inferred

Completions are also provided for a wide variety of specialized contexts including dimension names in [ and [[:

bracket

RStudio now provides completions for function arguments within function chains using magrittr’s %>% operator, for e.g. dplyr data transformation pipelines. Extending this behavior, we also provide the appropriate completions for the various ‘verbs’ used by dplyr:

dplyr        dplyr_verb

In addition, certain functions, such as library() and require(), expect package names for completions. RStudio automatically infers whether a particular function expects a package name and provides those names as completion results:

library

Completion is now also S3 and S4 aware. If RStudio is able to determine which method a particular function call will be dispatched to it will attempt to retrieve completions from that method. For example, the sort.default() method provides an extra argument, na.last, not available in the sort() generic. RStudio will provide completions for that argument if S3 dispatch would choose sort.default()

s3

Beyond what’s described above there are lots more new places where completions are provided:

  • For Shiny applications, completions for ui.R + server.R pairs
  • Completions for knitr options, e.g. in opts_chunk$get(), are now supplied
  • Completions for dynamic symbols within .C, .Call, .Fortran, .External

Additional Enhancements

Always On Completion

Previously RStudio only displayed completions “on-demand” in response to the tab key. Now, RStudio will proactively display completions after a $ or :: as well as after a period of typing inactivity. All of this behavior is configurable via the new completion options panel:

options

File Completions

When within an RStudio project, completions will be applied recursively to all file names matching the current token. The enclosing parent directory is printed on the right:

file

Fuzzy Narrowing

Got a completion with an excessively long name, perhaps a particularly long named Bioconductor package, or another variable or function name of long length? RStudio now uses ‘fuzzy narrowing’ on the completion list, by checking to see if the completion matches a ‘subsequence’ within each completion. By subsequence, we mean a sequence of characters not necessarily connected within the completion, so that for example, ‘fpse’ could match ‘file_path_sans_extension’. We hope that users will quickly become accustomed to this behavior and find it very useful.

fuzzy

Trying it Out

We think that the new completion features make for a qualitatively better experience of writing R code for beginning and expert users alike.  You can give the new features a try now by downloading the RStudio Preview Release.  If you run into problems or have feedback on how we could make things better let us know on our Support Forum.

 

Today we’re excited to announce htmlwidgets, a new framework that brings the best of JavaScript data visualization libraries to R. There are already several packages that take advantage of the framework (leaflet, dygraphs, networkD3, DataTables, and rthreejs) with hopefully many more to come.

An htmlwidget works just like an R plot except it produces an interactive web visualization. A line or two of R code is all it takes to produce a D3 graphic or Leaflet map. Widgets can be used at the R console as well as embedded in R Markdown reports and Shiny web applications. Here’s an example of using leaflet directly from the R console:

rconsole.2x

When printed at the console the leaflet widget displays in the RStudio Viewer pane. All of the tools typically available for plots are also available for widgets, including history, zooming, and export to file/clipboard (note that when not running within RStudio widgets will display in an external web browser).

Here’s the same widget in an R Markdown report. Widgets automatically print as HTML within R Markdown documents and even respect the default knitr figure width and height.

rmarkdown.2x

Widgets also provide Shiny output bindings so can be easily used within web applications. Here’s the same widget in a Shiny application:

shiny.2x

Bringing JavaScript to R

The htmlwidgets framework is a collaboration between Ramnath Vaidyanathan (rCharts), Kenton Russell (Timely Portfolio), and RStudio. We’ve all spent countless hours creating bindings between R and the web and were motivated to create a framework that made this as easy as possible for all R developers.

There are a plethora of libraries available that create attractive and fully interactive data visualizations for the web. However, the programming interface to these libraries is JavaScript, which places them outside the reach of nearly all statisticians and analysts. htmlwidgets makes it extremely straightforward to create an R interface for any JavaScript library.

Here are a few widget libraries that have been built so far:

  • leaflet, a library for creating dynamic maps that support panning and zooming, with various annotations like markers, polygons, and popups.
  • dygraphs, which provides rich facilities for charting time-series data and includes support for many interactive features including series/point highlighting, zooming, and panning.
  • networkD3, a library for creating D3 network graphs including force directed networks, Sankey diagrams, and Reingold-Tilford tree networks.
  • DataTables, which displays R matrices or data frames as interactive HTML tables that support filtering, pagination, and sorting.
  • rthreejs, which features 3D scatterplots and globes based on WebGL.

All of these libraries combine visualization with direct interactivity, enabling users to explore data dynamically. For example, time-series visualizations created with dygraphs allow dynamic panning and zooming:

NewHavenTemps

Learning More

To learn more about the framework and see a showcase of the available widgets in action check out the htmlwidgets web site. To learn more about building your own widgets, install the htmlwidgets package from CRAN and check out the developer documentation.

 

Follow

Get every new post delivered to your Inbox.

Join 19,145 other followers