Broadband-Testing Report: 

 

Runaware: An Online Software Evaluation Service

by Steve Broadhead

Tested

April 2001

Supplier

Runaware. www.runaware.com or Tel: +1-800-430-3231 US or +46 (0)8 463 1177 Europe

Table of Contents

Intro: Comparing Traditional Software Evaluations with the Runaware approach

Runaware: From The User Viewpoint – Online Evaluations For Free

Runaware: From The ISV Viewpoint – Creating A Virtual Marketplace And More

IQP: Interactive Qualification Process.

The Sponsored Partner Pavilion

Virtual Showroom

The Runaware Service: The Underlying Technology

Putting the Runaware evaluation Service to the test

The First-Time Experience – Registering With Runaware

Navigating Around The Runaware Site

The Menu System

The Business Pages

My Runaware

My Data

My Software

My History

Browsing Via Categories

Selecting The Recently Added/Most Popular Choices

Interactive Demonstration

Evaluating Applications

Evaluating Software For The First Time

At The End Of An Evaluation

Broadband-Testing’s Own Experiences

Overall Summary

From The End-user Perspective

From The ISV Perspective  

Overall 

Top

Intro: Comparing Traditional Software Evaluations with the Runaware approach

Who remembers when the majority of PC software applications used to arrive on a single floppy disk with a slim manual accompanying them?

Installations consisted of inserting the floppy in the drive, typing a:install or similar and two minutes later the installation was complete. Often the application itself was so simple that, within the hour you had completed the evaluation and could make a decision on whether to purchase it or not. Of course, you still had the problem of, firstly guaranteeing that your chosen software supplier had an evaluation copy of the application available and, secondly, then waiting for it to be delivered. But life, in general, was pretty simple. Apple Macintosh users would argue that it was even simpler than the scenario we’ve described!

But nothing lasts forever. As the computers themselves became more powerful, so the applications started to get bigger and more complex. Still on floppy disks, now the nightmares really began. Installations required so many floppy disk changes you not only spent often hours carrying out an installation but got severe aching arms in the process! And all for the sake of simply evaluating a piece of software you were not necessarily guaranteed to buy at the end of all that effort. And then it had to be taken off the PC again. The advent of the CD thankfully resolved the capacity problems of the floppy disk, often replacing the need for several of the latter with just one of the former.

However, still there was the issue of ordering the evaluation CD, awaiting its arrival, then carrying out the installation (and de-installation) and hoping there wouldn’t be any problems along the way. Even then, often the most you could hope to receive was a cut-down demo version of the software that did not provide you with a realistic means of appraising the product. Try to short-cut the process by purchasing a computer magazine with a CD attached and you were not only limited by what applications the magazine (typically based on the source of advertising revenue) had chosen to include, but you were absolutely restricted to demo versions or pure shareware.

Then along came the Internet and, with it, the option to download applications – again often in cut-down, demo format, but sometimes in fully featured format with an encoded time limit – for evaluation, from that vendors website. This approach is all well and good if you have high-bandwidth access to the Internet, so a 20-30 megabyte download becomes a feasible operation, rather than something likely to take several hours. If it ever completes successfully, that is… 

But for those with a 56K modem link, or even a 64K ISDN connection, such downloads are often prohibitively time consuming, especially when you do encounter problems and have to restart the whole download process, but then none of us have ever suffered that, have we! Moreover, again the problem exists where this process has to be repeated every time you want to evaluate a product. Where you only really want to take a quick look at a number of competing products, this option really doesn’t add up. Few people have the time, let alone the patience, required to spend the best part of a day downloading a number of products for evaluation.

So while the Internet has made it easier to evaluate software than before, the situation is still considerably less than ideal. What then, is the alternative? Here we’re looking at a new concept in software evaluating, being offered by Runaware: online evaluations, using the Internet as the access technology to a wide range of applications.

Top

 Runaware: From The User Viewpoint – Online Evaluations For Free

In an era where everyone seems to constantly bemoan a lack of time, any product or service that saves some of that valuable commodity has to be good news.

When it comes to evaluating new software products, often that lack of available time means evaluations are necessarily put on a back burner. This is clearly not good news for either the user seeking new and improved applications, nor the software developer in search of new customers, though the accountants may well be happy with the situation. So imagine instead, if the whole evaluation process was made into an instantaneous, online service. Suddenly evaluating software becomes something you can do during a lunch-break, or for 10-15 minutes at the end of each working day, for example.

Such is the idea behind Runaware – best described as an Internet-based online service for evaluating software applications on the fly. Runaware describes itself very specifically as an Evaluation Service Provider – we’ll forgive it the ESP joke - as opposed to an Application Service Provider (ASP). The difference is both subtle and very significant. Obviously, in order to provide the evaluation service in the first place it has to host the applications, but these are hosted on behalf of the Independent Software Vendor (ISV), not the consumer, as is the case with ASPs. An ASP exists primarily to host an end-user company’s applications on their behalf, which is a very different service to what Runaware offers.

Top

Runaware: In The Business Of Software Evaluation Not Reselling

Runaware is a privately held company based in Stockholm, Sweden. In 1999, Swedish entrepreneur and Runaware founder Pelle Axeland envisioned an electronic marketplace designed with the needs and problems of both software consumers and Independent Software Vendors (ISVs) in mind. The result was the service we’re evaluating here.  Runaware went live in May 2000. Software evaluations are available twenty-four hours a day, seven days a week and are free of charge.

An important point to make here is that Runaware is not a software reseller. You cannot purchase the software you evaluate directly from the company. However, every application Runaware hosts for evaluation has hyperlinks directly to related web-sites, often the vendors own, where you can acquire the software online. As such, it is the closest you can get to a one-stop shop approach while still remaining independent.

 

The key benefits of the Runaware service then, from the user viewpoint, are ease of evaluation and time saved compared with using traditional methods. No longer need you spend what may be hours downloading a demo version of an application from the Internet, or sending off for a demo CD then taking the time to install it when it finally arrives. Instead you simply go online to the Runaware web site and immediately start to evaluate a full-blown version of an application there and then. 

Better still, rather than having to go through that same tedious ritual every time you want to evaluate a different product, imagine being able to look at dozens of applications online in one session, without any kind of overhead. This is exactly what Runaware promises you can do. And it’s free of charge for evaluators. So does it sound too good to be true? Well maybe – so here we’re putting the system to the test to find out if Runaware really can deliver what sounds like the perfect online software evaluation service. 

There are, unsurprisingly, some limits with Runaware. While it is an online service that allows you to truly evaluate the full versions of software applications, a handful of functions are disabled. For example, print and save functions are not available and you cannot upload files to Runaware based applications. Nor is the selection of applications currently available anything other than a snapshot of the millions of software products available worldwide, though – that said - it is both relatively comprehensive and wide-ranging. 

Runaware’s long-term aim is that the software available for evaluation will also correspond to the needs of our users as those needs evolve, so users can actively help to direct the future shape of the Runaware service. As an individual uses the site, Runaware gathers information about users in a similar manner to an eCommerce site such as Amazon.com in order to help a user find the right software according to their demographics and needs. To assist with the evaluation, the site also provides links to reviews, feedback and other related information, so independent opinions of the software are readily available to compare with your own.

Top

Runaware: From The ISV Viewpoint – Creating A Virtual Marketplace And More

Naturally, any benefits to the end user looking to evaluate software are benefits to the ISV looking to sell its software to that user. In addition, however, Runaware promises many ISV-specific benefits to its service, which we’ll now run through. 

Runaware argues that a Runaware prospect referral is much more valuable than a traditional sales lead. There is clearly some truth in this. For starters, Runaware monitors the activity of the visitor on the site and documents their interaction with the titles they test and evaluate, including the amount of time spent with each application. The generated information regarding an ISVs products is available to only that ISV on secure pages on-line. 

This creates highly qualified sales prospect referrals that sales staff can use to grow their business without heavy sales and marketing spending. For example, if an ISV so wishes it can have Runaware automatically send it an e-mail when there is a prospect waiting for a call.

By definition, in creating an online source of evaluation software, Runaware has effectively created a true virtual marketplace – a kind of online exhibition. The company argues that its approach allows an ISV to eliminate both the costs of demo distribution and the delays in getting a new product launch jump-started. If Runaware hosts the applications on its site it means that prospects and customers have instant access to the product for evaluation. In turn this means instant access to prospects without the overhead of traditional marketing programs. 

The company does more than install and maintain ISV applications on a secure website – in itself a valuable service – but hosts these applications together with evaluation resources such as product information, white papers and tutorials, as well as hyperlinks directly to the ISVs web-site. 

 

Figure 1 – ISV Hyperlinked From The Runaware Site

In addition to the generic benefits for ISVs, Runaware also offers a number of optional services, which we’ll now outline.

Top

IQP: Interactive Qualification Process.

Runaware describes this as a process that automatically generates and qualifies sales leads and makes them instantly available to an ISV. So how does this work exactly? Well, in order to use Runaware, each visitor must register, providing a standard set of data including their name, e-mail address and country, as well as - when applicable – company name, size, job title and contact details. This “prospect profile” then goes into a database, private to that ISV, every time that ISVs software is tested. 

The IQP then measures each user’s level of interest as a result of the evaluation. It sorts information by user, session and application, including testing date & time, length of session, product information or user resources accessed, answers to on-line questions, surveys and polls, user comments and reviews, the user’s operating system, browser type & version. The IQP also allows an ISV to set criteria for qualifying what it regards as a “hot” prospect. Using these criteria, IQP sorts through responses and identifies interested users. 

Top

The Sponsored Partner Pavilion

Companies who offer widely used software platforms may choose to sponsor a “Pavilion” at the Virtual Marketplace so that all of their independent developer partners may exhibit their own applications. This environment carries the umbrella brand of the sponsor and provides high levels of exposure to the key development partners as well as the sponsor. 

In other words, it is the absolute online equivalent of sponsoring a pavilion at a computer exhibition with the same potential benefits. Visitors to Runaware find the pavilion prominently featured on its home page. A click on the logo will take them to the pavilion, which will feature virtual booths representing every ISV that has responded to the invitation to showcase its product.

The pavilion is designed, not only to increase partner sales, but to produce valuable qualified leads and market intelligence. User registration means that each time a potential customer tests a product, their profile and area of interest is entered in a database accessible only by that pavilion sponsor or their partners. In addition, discussion forums allow users to talk with each other and provide invaluable market feedback.

In order to make it easy for partners to join the pavilion, Runaware includes a webpage, accessible from the vendor web site containing all the tools and information necessary for ISVs to join the program. The vendor decides on the content and what ISV partners it wants to include. Both partner booths and the pavilion will be standardised with a consistent look and feel in line with the branding. Runaware also works out a mutual co-marketing plan in an attempt to generate maximum publicity and consumer interest.

Top

Virtual Showroom

Designed effectively a means of enabling publishers and developers to meet with prospects online, 24 hours a day, Runaware’s Virtual Showroom - designed to an ISVs own specifications - allows an ISV to offer the same convenience and software interaction as Runaware's “Virtual Marketplace” on its own website. 

It means an ISV can offer online testing and evaluation of all its software products in a private, secure online showroom powered by Runaware. The company claims this effectively and significantly upgrade web marketing efforts while lowering costs associated with offline product evaluation. It means that customers, channel partners, the sales force and any other relevant parties can access the showroom anywhere, anytime, so long as they have Internet access. It helps support a channel strategy by directing orders to an ISV or its reseller of choice, preserving control of purchase fulfilment. It could also arguably dramatically reduce time-to-market by allowing an ISV to publish the latest versions of its software during an early stage of production, or in some instances at a pre-production point.

Top

The Runaware Service: The Underlying Technology

Given the still far from perfect performance of the Internet, any attempt at providing an Internet-based, online software evaluation service clearly cannot be attempted lightly.

If the service is to sustain the interest of the user it must be sufficiently fast and reliable, as well as easy to use. With these requirements in mind, the Runaware team has developed a website that, while based on standard components, has been put together with special techniques for user handling and session handling, aimed at delivering the quality of services an ESP must provide. When developing the Runaware website, the focus was set on scalability, availability and speed. The basic premise was that downtime simply cannot be accepted since the site is globally accessed. At the same time, it should be possible to upgrade the site while it is up and running to ensure that regular updates can take please to maintain regular users’ interest. The site architecture has also been designed to supports several thousand concurrent software testers.

The starting point was with the hardware – based on Compaq servers with full failover capabilities. The front-end of the website consists of a special designed cluster solution that keeps track of users, statistics and web content. It also consists of a web enabling cluster solution that transfer the traffic from the software application being tested to the user browser. 

Figure 2 – The Runaware Architecture From User To Application

At the heart of the system is Galactica, a ''web-engine'' that sits 'between the web-server and the database. It covers many functions from site-administration to run-time security. By using Galactica, the development team can produce new functions and services for the Runaware website with a minimum of effort. Galactica is developed by Cross Communication and is currently used for large, information-intense websites as well as commercial websites.

The other key-component is Tarantella from SCO. It transfers the session-ata (software applications) directly from the application servers, for example CorelDraw, to the Java-applet in the user's browser, the PC acting as a “thin client”. 

The back-end consists of a number of application servers. These can be of different hardware platforms and operating systems. (NB - MacOS is currently not supported) This opens up the possibility to compare the same application on different platforms. For example, a Linux-version could easily be compared to the NT-version of the same software with the user deciding, online, which is the better implementation.

Overall, the Runaware site appears to have been designed with longevity and scalability in mind, which could be crucial to its long-term chances of success.

Top

Putting the Runaware evaluation Service to the test

The First-Time Experience – Registering With Runaware

The starting point for experiencing the Runaware site is straightforward enough – entering a URL of http://www.runaware.com into your browser. Runaware supports both Internet Explorer 4 or 5+ (recommended) and Netscape Navigator 4+.

Figure 3 – The Initial Runaware Home Screen

The home screen appears with a bright yellow button labelled join now! Clicking on this takes you to the registration process. This consists of a screen requesting basic information about you and your business – only some of which is mandatory, such as username and password – not unlike the registration process for any other online Internet service or free software download. Any errors you make are flagged so you can easily identify and correct them – for example, if the username you chose already exists, or if there are problems with the chosen password.

Figure 4 – The Runaware Registration Screen

Once complete, clicking on a button at the bottom of the page marked REGISTER completes the process. At this point you are returned to the home screen as a registered user with a welcome message and the complete functionality of Runaware now available. Future visits to the web-site from the same PC will be met with this customised screen so any changes you make to the personal Runaware configuration (see My Data description later) will be reflected automatically. Otherwise you will need to login with your member username and password, at which point your personal settings will be loaded.

Top

A Word For Macintosh Users

Officially, you cannot yet run Runaware on a Macintosh. This is due to a "known bug" in Apple's Java engine. Apple's tech support is working on it however; a new version will probably not be available until MacOS X according to Apple sources. Meantime, Runaware has found using Microsoft Internet Explorer 5.0 with Microsoft's Java Engine installed has proven successful.

When you register you have the option of subscribing to a free monthly newsletter, Runawareness. This is delivered by email and contains news of the company and new applications being hosted, complete with the respective URL. To evaluate the new application you simply have to click on the URL and your browser will load and navigate you directly to that application on the Runaware site for testing. We found this to be a very efficient way of discovering about, and evaluating, the new products as they appear on the Runaware site and would therefore recommend that any users subscribe to the newsletter when they register for the Runaware service.

Top

Navigating Around The Runaware Site

The Menu System

Once registered, all the features of the Runaware site are available. While these are primarily the applications available for evaluation, there are a number of supporting features which we should highlight here. Another important point to make at this point is that there are several ways to navigate the site; to load up an application or discover more information about a product, for example. What largely remains in place, regardless of the feature you’re accessing, is the basic menu structure as illustrated in Figure 3. The menu bar at the top of the page has five basic options: Home, About Runaware, FAQ, Demo Your Software and Link to us. The Home and About Runaware options are suitably obvious. Clicking on FAQ – Frequent Asked Questions - is a good starting point however, as it takes you to a kind of Q&A session, aimed at answering most of your basic questions about the Runaware service.

Figure 5 – The Runaware FAQ Screen

Clicking on Demo Your Software takes you to the section of the site which is primarily for ISVs – the Business Pages – which we’ll describe in more detail shortly and has its own “sub-menu” bar. The last option on the main menu bar, Link to us, takes you to a page providing three ways of linking Runaware to your own web site.

Figure 6 – The “Link to us” Screen

Here you can choose to join the affiliate program which can earn you commission for referrals which result in business for Runaware or its ISV customers, or provide a simple banner link between your web site and Runaware.com. In both cases a range of link button options are available to download. A third option for Runaware ISV customers is password protected and offers a range of link buttons for ISV use.

Top

The Business Pages

Typically, then, this menu format remains in place throughout the Runaware “experience”. However, clicking on Demo your software brings up a new submenu – the Runaware Business pages. While this section is intended primarily for ISVs (both existing and potential customers), it is also a home for useful extra and background information about Runaware, its services and its recent business dealings – such as technical alliances with other industry players – which many users may find of interest. For example, by clicking on Runaware Services you can find out more about the company, its services and the underlying technology. For a list of Runaware’s current ISV customers and their applications being hosted for evaluation, you just click on Customers and Partners

From the resulting screen it is then possible to go directly to the evaluation screen of any application by just clicking on its title in the customer list. This is just one of several ways to access the evaluation screens – the other ways we’ll cover shortly. Another unique part of the web site only accessible from the Business Pages is the Press center

Figure 7– The Press Center Within Business Pages

Here you can find a complete list of press releases, clippings from publications and – if you are a member of the press – useful material such as screenshots and board member photographs. To get back to the main menu you have to click on the www.runaware.com button as clicking on what is defined here as the Front page takes you only to the home page of the Business Pages section, which is a little confusing at first.

Top

My Runaware

Back at the “real” home page, there are a number of other options available, in addition to those we’ve already described. At the top right of the screen you’ll see three tabs labelled, My Data, My Software and My History. This is the area of the web-site that Runaware refers to as “My Runaware” and is a customisable area for registered users that reflect the choices you make and the software you evaluate. Briefly, each in turn can be described as follows:

Top

My Data

Once registered, the Runaware site makes use of your information to help guide you through the Runaware system. The information you provided during the registration process can be edited and appended at any time by clicking on the My Data tab. Your password can also be changed here. If you want to delete your account at Runaware you do so here, at which point all information about that account will be permanently deleted from the Runaware site.

Top

My Software

My Software is a list of your software favourites that you can build up as you evaluate products on the Runaware site. Every time you evaluate an application you are given the option to add it to My Software. You are also given the option to grade the product on a 1-5 scale and add personal comments, such as making a note of specific feature you liked or didn’t like, or had problems with. 

Do note these are private comments and are not made public. If you change your opinion following a further evaluation you can edit the grade and comments. From the list you can also quickly re-access the applications by simply clicking on the software title.

Figure 8 – My Software List

Top

My History

As you use the Runaware site, a log of application evaluations is automatically created, called My History. In this way it differs from My Software which is a personal list of software favourites. It shows your own previously evaluated programs in chronological order, telling you when you looked at the product. This is generated after each usage and stored for any future login so that you remember the specific titles. Clicking on a title takes you to the evaluation page for that piece of software, so this is another potential short-cut to an evaluation session.

Top

Browsing Via Categories

At all times during a session on the Runaware site you have the option of selecting an application for evaluation or obtaining further information on, by using the “Browse categories” box or the search function.

Figure 9 – Select Application By Browsing Categories

Thirteen categories are listed in alphabetical order from Administrative to Web Develop for Windows users and four categories for Linux users. At the top of the Browse Category box you can click on the specific tab for the operating system of your choice – Windows or Linux – to narrow down the category choice to those available for that specific operating system. Then, clicking on any of these categories brings up a list of the applications available within that category for evaluation. Clicking on a title at that point then takes you to the relevant evaluation page for that application. 

Top

Selecting The Recently Added/Most Popular Choices

Yet another way to select an application for evaluation is to look through the tab lists at the bottom of the page – the RECENTLY ADDED or MOST POPULAR lists. The former is self-explanatory while the latter consists of that week’s most popular applications for evaluation among Runaware users, with the previous week’s position in brackets, so you can see the relative popularity of each product, week by week. Bear in mind, however, that this is simply the number of users evaluating the product and not a recommendation as to the quality of the product in its own right. For this reason, you can add comments on any product via its evaluation page (see Evaluating Applications section).

Top

Interactive Demonstration

A completely different way of getting to know how to navigate around the Runaware site comes with the interactive demonstration option. Clicking on this box (Flash 4 is required on your PC and can be downloaded at this point if necessary) loads the demo program.

Figure 10 – Taking The Site Tour

Here you have the option of watching a fully automated site tour presentation, or selecting from a menu which aspect of the Runaware site you want to learn more about, by clicking on the relevant subject area. 

Figure 11 – Using The Interactive Demonstration Menu Options

It’s a sensible option to start with the tour and then work your way through the menu-based information before evaluating any software. Not that the site is complicated to use – it isn’t – but this way you’ll know all of the options available up front, rather than finding them by accident.

Top

Evaluating Applications

Evaluating Software For The First Time

Navigating your way around the Runaware site is all well and good but it is the software evaluations themselves that are key to the service. So just how successfully can you test an application, online, over the Internet?

The mechanics are very simple. Using any of the many ways to select an application we’ve already described in the report, you choose the software title you want to evaluate and are taken to the evaluation page for that product. 

Figure 12 – An Evaluation “Home” Page

At this point you have a number of options, which can vary depending on the product. To begin the evaluation you simply click on the RUN! button but before you do this there are various options available for getting background information on the product itself. For every product, Runaware provides a brief summary of the product as well as a summary from the vendors itself. In addition, depending on the product itself there may be hyperlinks to third party (IT press, for example) reviews and other background information such as white papers or even tutorials. These are all clearly marked under the Evaluation Resources/Other Information and What others say… sections of the page. You can also add your own comments under the What you say… section.  We found that most of the products we evaluated had extensive links to other web sites for both independent reviews and background information, so it is possible to be well clued up on the software before you actually get around to using it.

Once you are ready for the evaluation itself, clicking on the RUN! button opens up a new window with the selected application. As we mentioned earlier, no actual product installation takes place at this point, but the first time you use Runaware a Java applet is downloaded. This applet is approximately 1MB in size and will therefore require some downloading time from a modem connection. With our 54Kbps ISDN line, it took around two minutes to download. The important point to make here is that while this requires a small degree of patience first time around, once this applet has been installed on your computer, the operation will not need to be repeated. 

Figure 13 – Downloading The Java Applet On The First Runaware Session

 

Figure 14 – Downloading The Tarantella Software On The First Runaware Session

A minimal amount of user intervention is required at this point – accepting and agreeing to install the applet, for example. But on completing the Java applet download, the program you are evaluating loads up automatically without any other user intervention. Thereafter, on pressing RUN! to evaluate a product it will load up immediately.

Top

Netscape and Java security issues?

Using Netscape when launching a program generates two messages asking for your permission to download the Java client. You need to press [Grant] on both these messages. Also check the [Remember this decision] box if you do not want to see these messages the next time you use Runaware.  Please note: the owner of the security certificate is not Runaware but Santa Cruz Operations, Inc (SCO) since the client-software is a licensed product from SCO.

Top

At The End Of An Evaluation

To end an evaluation you simply close the window it is running in. At this point you are requested to answer some simple questions concerning the software you have just evaluated.

Figure 15 – Post-Evaluation Screens

First you are asked whether you found the software of interest or not? If the answer was “yes” a second window pops up giving you four options. These are:

  • To buy the software (by then routing you to the ISVs own site or to a relevant reseller on the Internet).

  • Whether you would like further information on the product.

  • Whether you are planning to buy the product but do not want to do so immediately. 

  • Simply to close the window and return to the Runaware site.

At this point the evaluation session ends but you can still choose to add your own comments about the product or add it to your My Software list before choosing another product to evaluate or leaving the Runaware site.

Top

Broadband-Testing’s Own Experiences

During the Runaware review period we ran through about a dozen evaluations, covering many different kinds of product such as graphics programs, network monitors, language translators and project management. Our PC clients used for the testing were both HP Vectra VL 400Mhz Pentium IIs with 128MB RAM, one running Windows 98 Second Edition and the second running Windows 2000 with IE5.5 as the browser. As mentioned earlier, our Internet access was via a 64Kbps ISDN connection. We made a point of accessing the Runaware site at many different times of day and night, including week-days and weekends, so any performance issues resulting from the Internet being congested could be both experienced and taken into account. 

Not surprisingly, performance did vary depending on when the session took place, but only once did we find the site too slow to be realistically usable. At best it was still a little like running a Windows application locally but where an extra few megabytes of RAM wouldn’t go amiss! In practice, this translates into a highly usable site for most of the applications we tried, at least for relatively short evaluation periods (10-20 minutes), though the heavier the graphics element of the evaluation product, the more patience was required. During most of the evaluation periods we experienced no problems, but we did encounter some errors – typically where the software refused to load a dummy data file locally for example (bearing in mind that you cannot upload any files from your PC) – on three occasions which Runaware is currently looking into. Of course, one problem with this test environment is that an error could equally be due to the ISVs own software being “buggy” which Runaware cannot reasonably take responsibility for! 

Top

Potential Issues with Firewalls and Proxy Servers

If you try an application, click RUN! and nothing happens it could be because your company or ISP is using a firewall or proxy. You may need to contact your Firewall administrator to possibly open Port 443 or if you use a proxy server the web proxy (8080) and SOCKS proxy (1080) ports should also be available. 

What we did like is that, barring the print, file save/upload functions not being available, this really is a complete online evaluation, even down to online tutorials within the ISVs applications, for example.

Figure 16 – Example Of Online Tutorial Within ISV Application

This is what makes the big difference between the Runaware service and online demo’s where only a limited subset of the functionality is typically available, at best. At worst, online demos are often little more than a glorified PowerPoint presentation. 

With Runaware you get basically the complete application to evaluate, there and then, with no download time apart from the first time you use the service. This is particularly gratifying when you test multiple applications in the one session. As such, in the course of a lunch hour, for example, you can readily compare two or three applications against each other. It really can potentially revolutionise the approach for software evaluations, albeit early days yet.

Top

Overall Summary

From The End-user Perspective

Runaware claims to be the first independent, free software evaluation “community”. Certainly, we know of no equivalent online evaluation service which is not directly tied to a software vendor. And it really does offer a lot more than any online demo can, naturally enough, given that it is the fully blown application you are evaluating.
The hook-line Runaware uses is: “All you need is a 56K modem”. Well we had the “luxury” of a 64K ISDN line during testing but performance was – overall – better than we had expected, given the quirks of Internet performance. The simplicity of the whole operation - no installing or downloading other than on the first occasion, little in the way of system requirements or compatibility issues to worry about – is very appealing and means very little effort is required in order to use the Runaware site. And it’s free!

Top

From The ISV Perspective

What Runaware offers is a completely new outlet for getting software in front of the masses. Those users who simply cannot or will not find the time to go hunting around web sites or fill in forms to request software evaluation packs can simply log onto the Runaware web site and test your software with a minimum of effort or time expended. The same ease of use concept applies to the ISV itself, with little or no effort required on its part to help host the evaluation site, though in this case there is a cost involved of course.

Beyond the basic evaluation service, Runaware does seem to be trying hard to put a complete portfolio of services together. So hosting the applications remotely may only be the starting point of a far more comprehensive and closer relationship between an ISV and Runaware. As an extension to existing sales and marketing operations, then, the Runaware service must be worth considering.

Top

Overall

As Runaware itself states, the site can best be compared to a trade show, except that it is permanently open and no travelling time is involved! At Broadband-Testing, for years we’ve been expounding the potential merits of the “virtual exhibition” and now Runaware has provided us with it. 

From our time testing the site, it is clear that it is not yet perfect – some errors did crop up – and the Internet is still a performance barrier in its own right, but overall – whether end-user or ISV - we highly recommend that you take a look at www.runaware.com .

 

Revised: February 06, 2007 .

Contact the  webmaster
with questions or comments about this web site.
 
Copyright © 2001-2007 Broadband-Testing

All rights reserved.