WO2001016757A1

WO2001016757A1 - A system and method for streaming data in java

Info

Publication number: WO2001016757A1
Application number: PCT/US2000/013766
Authority: WO
Inventors: Carl Ludewig
Original assignee: Audiobase, Inc.
Priority date: 1999-09-01
Filing date: 2000-05-19
Publication date: 2001-03-08
Also published as: AU5144500A

Abstract

A method is described for streaming data in Java comprising the steps of: storing audio/video data in a first addressable memory space (501); playing back the audio/video data stored in the first addressable memory space (501) when the first addressable memory space (501) is full; and storing additional audio/video data in a second addressable memory space (502) concurrently with the step of playing back audio/video data stored in the first addressable memory space (501). In addition, a system is described for streaming Java comprising: a first buffer (501) for storing a first portion of an audio/video data file; an audio/video player for playing back said first portion of said audio/video data file; and a second buffer (502) for storing a second portion of said audio/video data file at the same time that said audio/video player plays back said first portion of said audio/video data file.

Description

A SYSTEM AND METHOD FOR STREAMING DATA IN JAVA

BACKGROUND OF THE INVENTION Field of the Invention

This invention relates generally to the field of Java programming. More particularly, the invention relates to an improved system and method for streaming data in Java.

Description of the Related Art

Java

Java is an object-oriented programming language which has attracted considerable attention in recent years. One key feature of Java is that it can be used to design computer programs which are platform-independent. That is, the same Java object code may be used on numerous different operating systems including, for example, Windows 95, Unix, Solaris, and the Macintosh OS. This interoperability makes Java an ideal choice for programming Internet applications.

Once a program is written in Java source code, the Java compiler generates a compact, architecture-neutral object code known as a bytecode. Bytecodes can then be executed by a runtime interpreter residing on the client computer. This runtime interpreter is commonly referred to as a Java "virtual machine." The Java virtual machine reads each bytecode and inteφrets the program instructions which comprise that bytecode so that the instructions may be executed by the native microprocessor. Virtual machines are included in commonly available Internet browser applications such as Netscape Navigator™ or Microsoft Internet Explorer. ™

Java was derived from the popular C++ programming language and, as such, retained many significant features of that language. For example, Java, like C++, is object-oriented. Accordingly, Java programs are developed around "classes" and "objects." These two terms are not interchangeable but they are directly related to one another. A class can be thought of as a template or blueprint from which an object is made. When an object is created from a class, new object is called a new "instance" of that class. The object will initially have all of the same characteristics of the class from which it was derived. These characteristics are defined by the data defined within the object and the functions and procedures - i.e., the methods - associated with the object.

Streaming

Streaming is a technique for transmitting real time audio and/or video over a network. The puφose of streaming is to provide clients with uninterrupted audio/video content over a connection with limited bandwidth (e.g., a 56K modem connection). A few seconds of the digital audio/video stream is buffered at the client side before the signal is played back. This buffering technique compensates for momentary delays in packet delivery.

Although programs written in Java have become ubiquitous across the Internet, no built in streaming mechanism is provided in Java (i.e., streaming is not supported by the Java virtual machine). The only way to play back an audio (or video) file using Java will be described with reference to Figure 1. The Java application programming interface (hereinafter "API") is comprised of various software layers (i.e., classes) some of which are publicly available and others which are not. Using the publicly available Java API to play back an audio clip requires invoking a publicly available subroutine from one of the high level layers. For example, as illustrated in Figure 1, a call to "AudioClip.pl ay (URL)" will cause the audio file located at the designated Uniform Resource Locator (the address that defines a route to the file on the Web or other Internet facility; hereinafter "URL") to be downloaded and then played back on the client's machine.

There are several problems associated with this playback mechanism, however. First, more than one function call is required in the software configuration illustrated in Figure 1 to begin the process of downloading the audio file. For example, the call to "AudioClip.play(URL)" will trigger a call to an unpublished lower layer class "sun.audio.AudioPlayer.player.start" to start the audio download and playback process. Clearly, it would be more efficient (i.e., would save time and memory) in this case to be able to call the internal API function "sun.audio.AudioPlayer.player.startO" directly.

In addition, in the configuration illustrated in Figure 1, the entire audio file must be downloaded before playback of the file is initiated. This results in an unreasonably long delay between the time a particular audio file is selected and the time that playback begins. In fact, the foregoing playback mechanism is technically not "streaming" at all - it is simply file retrieval followed by file playback.

Another problem associated with this playback mechanism is that the retrieved file is uncompressed. No built-in audio/video compression mechanism exists in Java. Current audio and video compression codecs such as Adaptive Differential PCM (hereinafter "Adaptive Differential PCM") and MPEG Audio Layer 3 (hereinafter "MP3") allow an audio or video file to be compressed at a high compression ratio (e.g., 10: 1) with insubstantial loss to sound/video quality. Because the current, publicly available Java interface does not take advantage of any of these techniques, the time lag between the selection of the particular audio file and the playback of that file is far more attenuated than it needs to be.

Accordingly, what is needed is a system and method for performing streaming using Java. What is also needed is a system and method for streaming Java which incoφorates compression during audio/video playback. What is also needed is a system and method for streaming Java which can be implemented on currently existing Java platforms.

SUMMARY OF THE INVENTION

A method is described for streaming data in Java comprising the steps of: storing audio/video data in a first addressable memory space; playing back the audio/video data stored in the first addressable memory space when the first addressable memory space is full; and storing additional audio/video data in a second addressable memory space concurrently with the step of playing back audio/video data stored in the first addressable memory space.

In addition, a system is described for streaming Java comprising: a first buffer for storing a first portion of an audio/video data file; an audio/video player for playing back said first portion of said audio/video data file; and a second buffer for storing a second portion of said audio/video data file at the same time that said audio/video player plays back said first portion of said audio/video data file.

BRIEF DESCRIPTION OF THE DRAWINGS

A better understanding of the present invention can be obtained from the following detailed description in conjunction with the following drawings, in which: FIG. 1 illustrates a prior art system and method for playback of an audio file using Java.

FIG. 2 illustrates an exemplary network architecture used to implement elements of the invention.

FIG. 3 illustrates an exemplary computer architecture used to implement elements of the invention.

FIG. 4 illustrates one embodiment of a system for streaming audio or video in Java.

FIG. 5 illustrates dual addressable memory spaces (e.g. "streaming buffers") implemented in one embodiment of the system.

FIG. 6 illustrates a call to an unpublished Java class as implemented in one embodiment of the system.

FIG. 7 illustrates the crossfading of audio clips to create the auditory illusion of a continuous stream.

DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS A SYSTEM AND METHOD FOR STREAMING DATA IN JAVA

An improved system and method is described for streaming Java. In the following description, for the puφoses of explanation, numerous specific details are set forth in order to provide a thorough understanding of the invention. It will be apparent, however, to one skilled in the art that the invention may be practiced without some of these specific details. In other instances, well-known structures and devices are shown in block diagram form to avoid obscuring the underlying principles of the invention.

Embodiments of the invention include various steps, which will be described below. The steps may be embodied in machine-executable instructions. The instructions can be used to cause a general-puφose or special-puφose processor which is programmed with the instructions to perform certain steps. Alternatively, these steps may be performed by specific hardware components that contain hardwired logic for performing the steps, or by any combination of programmed computer components and custom hardware components.

Elements of the present invention may be provided as a computer program product which may include a machine-readable medium having stored thereon instructions which may be used to program a computer (or other electronic device) to perform a process. The machine-readable medium may include, but is not limited to, floppy diskettes, optical disks, CD-ROMs, and magneto-optical disks, ROMs, RAMs, EPROMs, EEPROMs, magnet or optical cards, propagation media or other type of media/machine-readable medium suitable for storing electronic instructions. For example, the present invention may be downloaded as a computer program product, wherein the program may be transferred from a remote computer (e.g., a server) to a requesting computer (e.g., a client) by way of data signals embodied in a carrier wave or other propagation medium via a communication link (e.g., a modem or network connection).

AN EXEMPLARY NETWORK ARCHITECTURE

Elements of the present invention may be included within a client-server based system 200 such as that illustrated in Figure 2. According to the embodiment depicted in Figure 2, a server 210 (which, depending on the particular embodiment, may include more than one server) communicates to a plurality of clients 230-235. The clients 230-235 may transmit and receive data from the server 200 over a variety of communication media including (but not limited to) a local area network 240 and/or a larger network 225 (e.g., the Internet). Alternative communication channels such as wireless communication via satellite broadcast (not shown) are also contemplated within the scope of the present invention.

The server 210 may include a database for storing Java code and/or digital audio and video data. The database may also store specific client data (e.g., information on how frequently a particular client logs in to server 210) and/or more general data. The database on the server 210 in one embodiment runs an instance of a Relational Database Management System (RDBMS), such as Microsoft™ SQL-Server, Oracle™ or the like.

A client may interact with and receive feedback from server 210 using various different communication devices and/or protocols. In one embodiment, the client logs in to server 210 via client software. The client software may include a browser application which supports Java such as Netscape Navigator™ or Microsoft Internet Explorer on the client's personal computer which communicates to server 210 via the Hypertext Transfer Protocol (hereinafter "HTTP"). In other embodiments included within the scope of the invention, clients may communicate with server 210 via cellular phones and pagers (e.g., in which the necessary software is embedded in a microchip), handheld computing devices, and/or touch-tone telephones. Server 210 may also communicate over a larger network (e.g., network 225) to other servers (e.g., server 250).

AN EXEMPLARY COMPUTER ARCHITECTURE

Having briefly described an exemplary network architecture which employs various elements of the present system and method, a computer system 300 representing exemplary clients 230-235; 260-261 and/or servers (e.g., server 210) in which elements of the system and method may be implemented will now be described with reference to Figure 3.

One embodiment of a computer system 300 comprises a system bus 320 for communicating information, and a processor 310 coupled to bus 320 for processing information. Computer system 300 further comprises a random access memory (RAM) or other dynamic storage device 325 (referred to herein as main memory), coupled to bus 320 for storing information and instructions to be executed by processor 310. Main memory 325 also may be used for storing temporary variables or other intermediate information during execution of instructions by processor 310. Computer system 300 also may include a read only memory (ROM) and/or other static storage device 326 coupled to bus 320 for storing static information and instructions used by processor 310.

A data storage device 327 such as a magnetic disk or optical disc and its corresponding drive may also be coupled to computer system 300 for storing information and instructions. Computer system 300 can also be coupled to a second I O bus 350 via and I O interface 330. A plurality of I/O devices may be coupled to I O bus 350, including a display device 343, an input device (e.g., an aphanumeric input device 342 and/or a cursor control device 341 ). For example, graphical depictions of various server 210 calculations and other types of data may be presented to the user on the display device 343.

The communication device 340 is for accessing remote servers, such as server 210, or other servers 250-253 via a network (e.g., the Internet 225). The communication device 340 may comprise a modem, a network interface card, or other well known interface device, such as those used for coupling to Ethernet, token ring, or other types of networks. In any event, in this manner, the computer system 300 may be coupled to a number of servers via a conventional network infrastructure, such as a company's local area network 240 and/or a larger network 225, for example.

ONE EMBODIMENT OF THE SYSTEM AND METHOD FOR STREAMING DATA IN JAVA

One particular embodiment of a system for streaming Java is illustrated in Figure 4. The streaming process is initiated when a client (e.g., client 235 of Figure 2) selects a URL which identifies audio or video content stored on a server (e.g., server 210). In one embodiment, the client 235 may not request the audio or video content directly. Instead, the client 235 may download a Web page from another server (e.g., server 250) which includes a tag that automatically loads the audio or video content from server 210. For example, the Web page from server 250 may have an ad tag inserted therein which automatically loads an advertisement from server 210 containing audio or video content. In either of the foregoing scenarios, however, the underlying principles of the system remain the same.

In one embodiment, to begin the streaming process once the audio/video content is identified, a call is initially made to a high level, published API class. This is represented in Figure 4 by the "URL.opensStream (URL)" call to a high level class. In response, as illustrated in Figure 5 at 510, the requested audio/video content is downloaded into one of a pair of addressable memory spaces or "streaming buffers" 501, 502 located on client 235 (in Figure 5 the portion of the buffer filled is highlighted). The pair of streaming buffers 501, 502 may be comprised of two memory spaces within a single contiguous physical memory or, alternatively, may be comprised of two independent physical memories. In one embodiment of the system, the downloaded content is decompressed/decoded on the fly before being loaded into the streaming buffers 501, 502. Particular codecs for accomplishing the decompression/decode process are described in more detail below.

Only after the first buffer 501 has been filled does the system begin to play back the audio/video content stored therein (the specific playback mechanism is described in detail below). Concurrently, as illustrated at 520 and 530, the system then begins to load the second of the two streaming buffers 502. Once the second of the two streaming buffers 502 is fully loaded at 540 the play/load functionality once again switches between the two buffers (i.e., Buffer 501 loads and Buffer 502 plays) and the process starts over.

In the worst case scenario in which playback of the first buffer 501 is completed before the second buffer 502 is filled playback will not initiate (in one embodiment of the system) until the second buffer 502 is completely filled. By filling one of the two buffers completely before initiating the playback mechanism, the present system insures that a predetermined amount of audio/video content will continually be available for playback (e.g., 5-10 seconds worth).

By contrast, current streaming systems use "circular buffers" in which a portion of the buffer is loaded before playback begins. However, if the buffer is loaded too slowly (e.g., because of a slow modem connection) all of the audio/video content stored in the buffer may be exhausted at the client side (i.e., the playback function may catch up to the buffer loading function). Thus, as soon as additional content is loaded into the buffer it will immediately be played back, resulting in disjointed audio/video playback or incoherent playback if the modem connection is slow enough.

In one embodiment of the system, the audio/video playback is initiated by making a function call to an internal (unpublished) Java class. For example, once the first of the two buffers has been fully loaded, a call may be made to the low-level unpublished function "sun.audio.AudioPlayer.player.startO" as illustrated in Figure 6 to begin playing back the contents of the first buffer while the second buffer is loaded with additional audio/video content. The dotted lines in Figure 6 indicate the previous manner in which the sun.audio.AudioPlayer.player.start() function was accessed (via the published "AudioClip.playO" function). Thus, streaming may be accomplished on currently available Java platforms by initiating the download of a particular piece of audio/video content to a pair of streaming buffers using a known (i.e., public) Java function ("URL.openStream (URL)") and subsequently invoking a low level, unpublished Java media playback function within an unpublished Java function ("sun.audio.AudioPlayer.player.startO") to play back the content from the pair of streaming buffers. As illustrated in Figure 7, another embodiment of the system includes a technique referred to herein as "clip crossfading" for playing back audio content. This embodiment may be implemented either with the publicly available Java playback mechanism or with the unpublished playback mechanism described above.

Clip crossfading is used to reduce the problem of audible clicking which sometimes occurs when switching playback from one buffer 501 to another 502. As illustrated, in order to solve this clicking problem the playback from the first buffer 501 is faded out starting at time tl while the playback from the second buffer 502 is faded in. Accordingly, in this embodiment, each buffer will contain a small amount of the same audio content (i.e., to be played concurrently during the fading process). At time t2, the operation of the two buffers will once again switch and buffer 502 will fade out as buffer 501 fades in. This clip crossfading technique effectively reduces or removes the problem of clicking associated with switching between playback buffers.

It should be noted that decoding of the downloaded audio/video content may occur either before or after the content is loaded into the streaming buffers depending on the particular embodiment. Moreover, numerous different codecs may be used in the present system to encode and decode the audio/video content. In one embodiment of the system, a unique version of the Adaptive PCM codec (hereinafter "ADPCM") is used. More specifically, a version of ADPCM developed by Sun Microsystems™ is modified to run more efficiently on a Java platform (i.e., Java virtual machine). The particular ADPCM version is one which was developed by Sun to take advantage of various features of hardware 2's compliment arithmetic. However, many of these features were implemented using complex functions which were not suited for execution on a Java platform.

For example, one ADPCM optimization is to remove certain functions and modules which are unnecessary to the underlying decoding functionality of the system. Removing extraneous modules in this manner is particularly important in a Java environment because of the need to produce compact programs (which may be downloaded efficiently over sometimes slow Internet connections). Accordingly, in one embodiment of the system, all decoder postprocessing functions such as decoder post-processing function "tandem_ulaw" (originally used to tie together various decoder functions/processes), are extracted from the original ADPCM codec. A second optimization used in one embodiment of the system is to remove and/or modify functions and modules which were optimized to run on a particular type of operating system and microprocessor. For example, in its codec, SUN incorporated a complex function for multiplying two numbers which established a custom floating point number format. To simplify the multiplication step (and thereby decrease the size of the underlying program) the SUN multiplication function was replaced by a multiplication function optimized to run more efficiently on a Java platform. More specifically, the custom floating point data type was replaced with a primitive (i.e., established) floating point data type.

In one embodiment of the system, additional Java optimization is accomplished by including a bit-shifting module which simplifies ADPCM multiplications and divisions (e.g., the ADPCM "fmult" function). Bit shifting is particularly useful when an operation is to be performed using a power of two. For example, multiplying the numbers 9 and 8 in binary format requires nothing more than the step of adding three zeros to the binary equivalent of 9. Accordingly, 8 ( 2¹ = 100 binary) x 9 (101 binary) = 72 (101000 binary). This bit shifting technique is analogous to adding zeros to decimal numbers when multiplying or dividing by a power of 10 (e.g., 204 x 100 = 20,400). It significantly improves the efficiency of the underlying decompression codec because the step of adding zeros is simpler than multiplying two values.

In one particular embodiment, the entire Sun ADPCM "fmult" function is replaced with a multiplication using primitive data types and bit shifting operations. Accordingly this embodiment reduces the overhead associated with making an additional function call ("fmult") and reduces the complexity of the underlying mathematical calculations.

In one embodiment of the system, when a client 235 requests audio/video content from a server 250, the content will be transmitted along with the compression codec and the Java applet used to play back the content. Accordingly, in this embodiment, the user does not need a particular plug-in (e.g., RealNetworks' RealPlayer™) installed in his web browser to play back audio/video content. Rather, a small, portable player is transmitted in the form of a Java applet with each client request, regardless of the plug-in applications installed on the client's 235 's web browser. An improved system and method for streaming data in Java has been described. In the foregoing description, for the purposes of explanation, numerous specific details were set forth in order to provide a thorough understanding of the system and method. It will be apparent, however, to one skilled in the art that the system and method may be practiced without some of these specific details. In other instances, well-known structures and devices were illustrated in block diagram form to avoid obscuring the underlying principles of the system and method.

Claims

CLAIMSWhat is claimed is:

1. A method for streaming data in Java comprising the steps of: downloading a first portion of audio/video data from a server and storing said first portion of audio/video data in a first addressable memory space; playing back said first portion of audio/video data stored in said first addressable memory space after said first addressable memory space is filled with said first portion of audio/video data; and storing a second portion of said audio/video data in a second addressable memory space concurrently with said step of playing back said first portion of audio/video data stored in said first addressable memory space.

2. The method as claimed in Claim 1 including the additional step of playing back said second portion of audio/video data stored in said second addressable memory space when said second addressable memory space is full.

3. The method as claimed in Claim 2 including the addition step of storing a third portion of said audio/video data in said first addressable memory space concurrently with playing back said second portion of audio/video data stored in said second addressable memory space.

4. The method as claimed in Claim 1 wherein an unpublished Java function is invoked to play back said portions of audio/video data.

5. The method as claimed in Claim 4 wherein said unpublished Java function is "sun.audio.AudioPlayer.player.startO."

6. The method as claimed in Claim 4 wherein a published Java function is invoked to download said audio/video data to be stored in said first addressable memory space.

7. The method as claimed in Claim 6 wherein said published Java function is "URL.openStream ()."

8. The method as claimed in Claim 1 wherein said audio/video data is decompressed/decoded before being played back.

9. The method as claimed in Claim 8 wherein said audio/video data is decompressed/decoded before being loaded into said first addressable memory space.

10. The method as claimed in Claim 9 wherein said decompression is accomplished with an Adaptive PCM compression function.

1 1. The method as claimed in claim 10 wherein said Adaptive PCM compression function is modified to run more efficiently in Java.

12. The method as claimed in claim 1 1 wherein said modification includes replacing custom data types with primitive data types.

13. The method as claimed in claim 11 wherein said modification includes replacing multiplication and division functions with bit shifting operations.

14. A method for streaming audio/video content in Java comprising the steps of: making a call to a published Java class to download audio/video content to a first addressable memory space; and making a call to an unpublished Java class to play back said audio/video content when a predetermined amount of said first addressable memory space has been filled with said downloaded audio/video content.

15. The method for streaming audio/video content in Java as claimed in claim 14 further including the steps of making a call to said published Java class to download audio/video content to a second addressable memory space while said audio/video content is played back from said first addressable memory space.

16. The method for streaming audio/video content in Java as claimed in claim 14 wherein said published Java function is "URL. openStream()."

17. The method for streaming audio/video content in Java as claimed in claim 14 wherein said unpublished Java function is "sun.audio.AudioPlayer. player. startQ."

18. The method for streaming audio/video content in Java as claimed in claim 14 wherein before being played back, said audio/video content is decompressed.

19. The method for streaming audio/video content in Java as claimed in Claim 18 wherein said decompression is accomplished with an Adaptive PCM compression function.

20. The method for streaming audio/video content in Java as claimed in claim 19 wherein said Adaptive PCM compression function is modified to run more efficiently in Java.

21. The method for streaming audio/video content in Java as claimed in claim 20 wherein said modification includes replacing custom data types with primitive data types.

22. The method for streaming audio/video content in Java as claimed in claim 20 wherein said modification includes replacing multiplication and division functions with bit shifting operations.

23. A system for streaming audio/video content comprising: a server which transmits a Java applet to a client, said Java applet, making a call to a published Java class to download audio/video content from said server to a first addressable memory space on said client; and making a call to an unpublished Java class to play back said audio/video content when a predetermined amount of said first addressable memory space has been filled with said downloaded audio/video content.

24. The system for streaming audio/video content in Java as claimed in claim 23 wherein said Java applet performs the additional functions of making a call to said published Java class to download audio/video content from said server to a second addressable memory space while said audio/video content is played back from said first addressable memory space.

25. The system for streaming audio/video content in Java as claimed in claim 23 wherein said published Java function is "URL. openStream()."

26. The system for streaming audio/video content in Java as claimed in claim 23 wherein said unpublished Java function is "sun. audio. AudioPlayer. player.start()."

27. The system for streaming audio/video content in Java as claimed in claim 23 wherein before being played back, said audio/video content is decompressed.

28. The system for streaming audio/video content in Java as claimed in claim 23 wherein said decompression is accomplished with an Adaptive PCM compression function.

29. The system for streaming audio/video content in Java as claimed in claim 23 wherein said Adaptive PCM compression function is modified to run more efficiently in Java.

30. The system for streaming audio/video content in Java as claimed in claim 23 wherein said modification includes replacing custom data types with primitive data types.

31. The system for streaming audio/video content in Java as claimed in claim 23 wherein said modification includes replacing multiplication and division functions with bit shifting operations.