AU2017228700A1 - System and method of rendering a surface - Google Patents

Abstract

SYSTEM AND METHOD OF RENDERING A SURFACE A system and method of rendering a surface into a plurality of output pixels. The method comprises receiving a description of the surface (1210); for each of the plurality of output pixels, receiving a micro-structure description of variations of material appearance on the surface within said pixel, wherein the micro-structure description is associated with a plurality of representative micro-structure faces, each of the plurality of micro-structure faces being associated with one of at least a primary material appearance and a secondary material appearance (1220); and determining, for each of the plurality of output pixels, a visibility measure for each visible representative micro-structure face based on a normal of the surface at the position of the pixel, a viewing direction and the corresponding micro-structure description, the normal of the surface being determined from the received description of the surface (1230). The method furhter comprises determining, for each of the plurality of output pixels, an aggregate material appearance for said pixel by combining the primary material appearance and the secondary material appearance, the contribution of each of the primary and secondary material appearances weighted based on the determined visibility measures of the visible representative micro-structure faces (1240); and rendering each of the output pixels using the determined aggregate material appearance to render the surface (1250). 13621053_1 (Wide-Area) Communications Network 120 Microphone 124160 117 (Local-Area) Video Communications Display EtNetwork 122 114 Mxtm 123 MoHDD 110 J9 PrdocVieor 1/ Interface Moa emor ptica DiSkoag 105HD 11116Drv 112 FKeyboard 0 Scanner 126 Disk Storage Camer 12=7103 Medium FIG. 1A 13621054_1

Description

2017228700 15 Sep 2017

SYSTEM AND METHOD OF RENDERING A SURFACE

TECHNICAL FIELD [0001] The present invention relates to a method of and an apparatus for simulating and displaying the surface appearance of a material on a computer monitor.

BACKGROUND [0002] Appearance characteristics of some materials can change depending on a viewing angle. For example, colour travel is a property of a material where the apparent colour of the material observed by a viewer depends on the viewing angle. The colour of the material is said to “travel” to a different colour as the viewing angle changes. Physical materials can be fabricated to exhibit colour travel using a number of different fabrication methods.

[0003] Fabrication of materials having colour travel can be expensive and time-consuming. Therefore is it desirable for an operator or designer to be able to accurately preview on a computer system the colour travel effect of a material before the material is fabricated. The accuracy of the preview in relation to the final fabricated material allows the operator to correctly alter the material properties to meet any relevant specification criteria before fabrication, avoiding wasted fabrications and saving money. It is also desirable for the operator to alter the properties of the material at interactive frame-rates, such that the preview is updated with the altered properties in real-time. Updating the preview in real-time shortens the iteration time between successive refinements of the material properties by the operator, saving time.

[0004] One known method models the physical details of the colour travel fabrication method using explicit three dimensional (3D) geometry on the surface of the model. The method using explicit 3D geometry precludes real-time use however. Using explicit 3D geometry provides a preview that is accurate, but the amount of 3D geometry required to render the preview is so large that only a very small patch of the material can be rendered at interactive frame rates. Rendering only a very small patch of the material at interactive frame rates fails to give the operator a suitable idea of what the entire fabricated material will look like.

[0005] Another known method attempts to generate a preview suitable for real-time use at a cost of providing inaccurate representation of the fabricated material. One example models colour travel using only two colours and the viewing angle that is required to view the two

13621053_l

2017228700 15 Sep 2017 colours, i.e. for a particular viewing angle a particular colour is determined in advance. Determining the colour in advance provides a preview that can be rendered at interactive framerates, but the colour travel model is typically overly simplistic, resulting in various properties that are present in the fabricated material, but not the preview. The method also fails to give the operator a suitable idea of what the fabricated material will look like.

[0006] The known methods fail to accurately render a preview of a material with a material appearance travel effect at interactive frame-rates.

SUMMARY [0007] It is an object of the present invention to substantially overcome, or at least ameliorate, one or more disadvantages of existing arrangements.

[0008] One aspect of the present invention provides a method of rendering a surface into a plurality of output pixels, the method comprising: receiving a description of the surface; for each of the plurality of output pixels, receiving a micro-structure description of variations of material appearance on the surface within said pixel, wherein the micro-structure description is associated with a plurality of representative micro-structure faces, each of the plurality of micro-structure faces being associated with one of at least a primary material appearance and a secondary material appearance; determining, for each of the plurality of output pixels, a visibility measure for each visible representative micro-structure face based on a normal of the surface at the position of the pixel, a viewing direction and the corresponding micro-structure description, the normal of the surface being determined from the received description of the surface; determining, for each of the plurality of output pixels, an aggregate material appearance for said pixel by combining the primary material appearance and the secondary material appearance, the contribution of each of the primary and secondary material appearances weighted based on the determined visibility measures of the visible representative micro-structure faces; and rendering each of the output pixels using the determined aggregate material appearance to render the surface.

[0009] According to another aspect, the micro-structure description comprises one of a) a set of normals for a micro-structure geometry, b) a set of vertices for the representative microstructure geometry, c) a set of normals and faces for the micro-structure geometry, and d) a look-up table.

13621053_l

2017228700 15 Sep 2017 [0010] According to another aspect, the primary material appearance relates to a primary perceived colour of the material and the secondary material appearance relates to a secondary perceived colour of the material.

[0011] According to another aspect, the visibility measures are determined using normals of the micro-structure faces associated with the output pixel, the normals being determined from the micro-structure description and the normal of the surface.

[0012] According to another aspect, the method further comprises: determining visible faces of the micro-structure faces using normals of the micro-structure faces and the viewing direction; determining a relative contribution of each micro-structure face to appearance of the output pixel compared to contribution of other visible faces using normals of the visible faces and the viewing direction.

[0013] According to another aspect, the visibility measures are further based on a relative area of each micro-structure face.

[0014] According to another aspect, the visibility measure is determined at run-time by performing an off-screen render of the micro-structures at the viewing angle and determining a relative contribution of each micro-structure face that is visible to appearance of the output pixel.

[0015] According to another aspect, off-screen renders of the micro-structures are performed before run-time for a plurality of representable viewing angles to determine corresponding visibility measures, the visibility measures being stored in a look-up table.

[0016] According to another aspect, determining the visibility measure relates to interpolating values in the look-up table.

[0017] According to another aspect, the micro-structure relates to one or more of a pyramid, an inverted pyramid, truncated pyramid, cylinder, shapes with self-occlusion, and shapes with overhangs or holes.

[0018] According to another aspect, a distribution of normals of the representative microstructure faces describes a shape with at least one curved surface.

13621053_l

2017228700 15 Sep 2017 [0019] According to another aspect, a distribution of rotated face normals for each representative micro-structure faces approximates a micro-structure shape with at least one curved surface.

[0020] According to another aspect, the micro-structures are spatially distributed according to a micro-structure distribution value.

[0021] According to another aspect, a bi-directional reflectance distribution function (BRDF) of the material is assigned to spaces between the micro-structures.

[0022] According to another aspect, a micro-structure description associated with one of the output pixels differs from a micro-structure description associated with another of the output pixels.

[0023] According to another aspect, the rendering comprises: modifying a reflectance distribution of a material using properties of an area-based light source; and rendering the surface under illumination of the area-based light source using the modified reflectance distribution in accordance with a point light source rendering method.

[0024] According to another aspect, the rendering comprises: simulating illumination of the surface by an area-based light source by modifying a reflectivity distribution associated with a material to represent reflectivity of the material under illumination of the area-based light source, modifying the reflectivity distribution comprising deriving a distribution with a separate control of a steepness and a width of the distribution, wherein the steepness of the derived distribution is based on the roughness of the material and the width of the derived distribution is based on a size of the light source; and rendering the surface in accordance with a point light source rendering method using the modified reflectivity distribution to generate a rendered output, wherein the rendered output simulates illumination of the material by the area-based light source.

[0025] Another aspect of the present disclosure provides a system comprising: a memory; and a processor, wherein the processor is configured to execute code stored on the memory for implementing a method of rendering a surface into a plurality of output pixels, the method comprising: receiving a description of the surface; for each of the plurality of output pixels, receiving a micro-structure description of variations of material appearance on the surface

13621053_l

2017228700 15 Sep 2017 within said pixel, wherein the micro-structure description is associated with a plurality of representative micro-structure faces, each of the plurality of micro-structure faces being associated with one of at least a primary material appearance and a secondary material appearance; determining, for each of the plurality of output pixels, a visibility measure for each visible representative micro-structure face based on a normal of the surface at the position of the pixel, a viewing direction and the corresponding micro-structure description, the normal of the surface being determined from the received description of the surface; determining, for each of the plurality of output pixels, an aggregate material appearance for said pixel by combining the primary material appearance and the secondary material appearance, the contribution of the each of the primary and secondary material appearances weighted based on the determined visibility measures of the visible representative micro-structure faces; and rendering each of the output pixels using the determined aggregate material appearance to render the surface.

[0026] Another aspect of the present disclosure provides a non-transitory computer readable medium having a computer program stored thereon to implement a method of rendering a surface into a plurality of output pixels, the method comprising: receiving a description of the surface; for each of the plurality of output pixels, receiving a micro-structure description of variations of material appearance on the surface within said pixel, wherein the micro-structure description is associated with a plurality of representative micro-structure faces, each of the plurality of micro-structure faces being associated with one of at least a primary material appearance and a secondary material appearance; determining, for each of the plurality of output pixels, a visibility measure for each visible representative micro-structure face based on a normal of the surface at the position of the pixel, a viewing direction and the corresponding micro-structure description, the normal of the surface being determined from the received description of the surface; determining, for each of the plurality of output pixels, an aggregate material appearance for said pixel by combining the primary material appearance and the secondary material appearance, the contribution of the each of the primary and secondary material appearances weighted based on the determined visibility measures of the visible representative micro-structure faces; and rendering each of the output pixels using the determined aggregate material appearance to render the surface.

[0027] Another aspect of the present disclosure provides apparatus configured to render a surface into a plurality of output pixels, comprising: a memory; a processor configured to execute code stored on the memory to: receive a description of the surface; for each of the

13621053_l

2017228700 15 Sep 2017 plurality of output pixels, receive a micro-structure description of variations of material appearance on the surface within said pixel, wherein the micro-structure description is associated with a plurality of representative micro-structure faces, each of the plurality of micro-structure faces being associated with one of at least a primary material appearance and a secondary material appearance; determine, for each of the plurality of output pixels, a visibility measure for each visible representative micro-structure face based on a normal of the surface at the position of the pixel, a viewing direction and the corresponding micro-structure description, the normal of the surface being determined from the received description of the surface;

determine, for each of the plurality of output pixels, an aggregate material appearance for said pixel by combining the primary material appearance and the secondary material appearance, the contribution of the each of the primary and secondary material appearances weighted based on the determined visibility measures of the visible representative microstructure faces; and render each of the output pixels using the determined aggregate material appearance to render the surface.

BRIEF DESCRIPTION OF THE DRAWINGS [0028] One or more example embodiments of the present invention will now be described with reference to the following drawings, in which:

[0029] Figs. 1A and IB collectively form a schematic block diagram of a general purpose computer on which the arrangements described may be practised;

[0030] Fig. 2 shows colour change that is apparent when a surface with a colour travel material is rotated 90 degrees relative to a viewer;

[0031] Fig. 3shows a physical implementation of a colour travel material where the surface is covered in pyramid micro-structures with different material appearances on different pyramid sides;

[0032] Fig. 4shows a mesh representation of an input surface geometry to be rendered, with a normal defined at each vertex;

[0033] Fig. 5 shows light reflectance characteristics described by an example bidirectional reflection distribution function (BRDF) relative to a number of viewing points;

13621053_l

2017228700 15 Sep 2017 [0034] Fig. 6 shows a pyramid micro-structure, showing orientation to the local surface normal, and with normals assigned to micro-faces;

[0035] Fig. 7 shows a pyramid micro-structure parameterised by the height of the pyramid relative to width;

[0036] Fig. 8 illustrates results of a visibility function operating on a pyramid micro-structure at different orientations to the viewing direction;

[0037] Fig. 9 shows a truncated square pyramid micro-structure, showing orientation to the local surface normal, and with normals assigned to micro-faces;

[0038] Fig. 10 shows a truncated square pyramid micro-structure parameterised by both the height of the non-truncated square pyramid relative to width, and the height of the truncation relative to the pyramid width;

[0039] Fig. 11 shows examples of other micro-structures that can be used to implement a colour travel effect;

[0040] Fig. 12 is a schematic flow diagram showing an example method of rendering a surface to a plurality of output pixels;

[0041] Fig. 13 is a schematic flow diagram showing input and output parameters of a visibility function;

[0042] Fig. 14 is a schematic flow diagram showing a method of rendering an input surface geometry with a material appearance into an output pixel buffer;

[0043] Fig. 15 is a schematic flow diagram showing a method of implementing a visibility function; and [0044] Fig. 16 is a schematic flow diagram showing a method of rendering colour travel.

13621053_l

2017228700 15 Sep 2017

DETAILED DESCRIPTION INCLUDING BEST MODE [0045] Where reference is made in any one or more of the accompanying drawings to steps and/or features, which have the same reference numerals, those steps and/or features have for the purposes of this description the same function(s) or operation(s), unless the contrary intention appears.

[0046] Colour travel is a property of a material occurring where the apparent colour of the material depends on the viewing angle. That is, as a viewer moves relative to a surface geometry on which the material is exhibited, the apparent colour at a particular location on the surface geometry changes to a different apparent colour. It is said that the colour at that location “travels” to the different colour. Colour travel provides one example of material appearance travel, that is a material appearance that varies based on viewing angle.

[0047] Fig. 2a demonstrates the colour travel effect on a square surface geometry 210. The square surface geometry exhibits a material with a colour travel effect, that is a colour travel material. The surface geometry 210 has an identifying mark 220 to indicate the orientation of the surface geometry when viewed from different viewpoints. Two example viewpoints are indicated as 230 240.

[0048] Fig. 2b illustrates an appearance of the colour travel material on the surface geometry 210 when viewed from the first viewpoint 230. The material on the surface geometry has a certain apparent colour 250.

[0049] Fig. 2c illustrates the appearance of the colour travel material on the surface geometry 210 when viewed from the second viewpoint 240. The material on the surface geometry has a certain apparent colour 260, different to the apparent colour 250.

[0050] One way that a colour travel effect can be achieved on a physical surface geometry is to cover the surface geometry with a material of micro-structures with specific characteristics. Fig. 3 shows a surface geometry 300 covered with a plurality of micro-structure pyramids 310. Each pyramid 310 has a side that has a particular material appearance 320 and a different side that has a different material appearance 330. The appearances 320 and 330 or a combination thereof is perceived by a viewer depending on the viewing angle. Other sides of the micro-structures 310 may also have different material appearances. The pyramids 310 are aligned such that viewing the surface geometry from a particular angle will show predominantly one material

13621053_l

2017228700 15 Sep 2017 appearance, while viewing the surface geometry from a different angle will show predominantly a different material appearance. Importantly, the scale of the micro-structures 310 is smaller than the optical resolution of the viewer. Thus the individual micro-structures cannot easily be visually resolved by the viewer, and the colour travel effect dominates. For ease of reference, the description below relates to a colour travel effect, that is changes of the apparent colour with changes in the viewing angle. However, other observed properties or characteristics of a material can vary by changing a viewing angle. The arrangements described can relate to material appearance characteristics that can vary when viewed from different angles. Material appearance characteristicscan be represented via parameters of a BRDF function, such as roughness, reflectivity or anisotropy. The arrangements described relate to material appearance travel, the colour travel effect being an example of the material appearance travel effect.

[0051] The material covering each of various sides of the micro-structure pyramids 310 may be a single colour material, or a more complicated material that has colour or other material properties that vary spatially over the surface geometry. The values of the colours or material properties can be stored in textures that map onto the surface geometry.

[0052] When fabricating a physical surface geometry with a material having appearance travel characteristics, a computer generated rendering is desired to function as a preview of the fabricated material. Viewing the preview allows a designer, an operator of the computer, to make informed decisions about the properties of the material, potentially altering the properties to ensure that the final fabricated surface meets the desired specifications.

[0053] Figs. 1A and IB depict a general-purpose computer system 100, upon which the various arrangements described can be practiced.

[0054] As seen in Fig. 1A, the computer system 100 includes: a computer module 101; input devices such as a keyboard 102, a mouse pointer device 103, a scanner 126, a camera 127, and a microphone 180; and output devices including a printer 115, a display device 114 and loudspeakers 117. An external Modulator-Demodulator (Modem) transceiver device 116 may be used by the computer module 101 for communicating to and from a communications network 120 via a connection 121. The communications network 120 may be a wide-area network (WAN), such as the Internet, a cellular telecommunications network, or a private WAN. Where the connection 121 is a telephone line, the modem 116 may be a traditional

13621053_l

2017228700 15 Sep 2017 “dial-up” modem. Alternatively, where the connection 121 is a high capacity (e.g., cable) connection, the modem 116 may be a broadband modem. A wireless modem may also be used for wireless connection to the communications network 120.

[0055] The computer module 101 typically includes at least one processor unit 105, and a memory unit 106. For example, the memory unit 106 may have semiconductor random access memory (RAM) and semiconductor read only memory (ROM). The computer module 101 also includes an number of input/output (VO) interfaces including: an audio-video interface 107 that couples to the video display 114, loudspeakers 117 and microphone 180; an I/O interface 113 that couples to the keyboard 102, mouse 103, scanner 126, camera 127 and optionally a joystick or other human interface device (not illustrated); and an interface 108 for the external modem 116 and printer 115. In some implementations, the modem 116 may be incorporated within the computer module 101, for example within the interface 108. The computer module 101 also has a local network interface 111, which permits coupling of the computer system 100 via a connection 123 to a local-area communications network 122, known as a Local Area Network (LAN). As illustrated in Fig. 1 A, the local communications network 122 may also couple to the wide network 120 via a connection 124, which would typically include a so-called “firewall” device or device of similar functionality. The local network interface 111 may comprise an Ethernet circuit card, a Bluetooth® wireless arrangement or an IEEE 802.11 wireless arrangement; however, numerous other types of interfaces may be practiced for the interface 111.

[0056] The I/O interfaces 108 and 113 may afford either or both of serial and parallel connectivity, the former typically being implemented according to the Universal Serial Bus (USB) standards and having corresponding USB connectors (not illustrated). Storage devices 109 are provided and typically include a hard disk drive (HDD) 110. Other storage devices such as a floppy disk drive and a magnetic tape drive (not illustrated) may also be used. An optical disk drive 112 is typically provided to act as a non-volatile source of data. Portable memory devices, such optical disks (e.g., CD-ROM, DVD, Blu-ray Disc™), USB-RAM, portable, external hard drives, and floppy disks, for example, may be used as appropriate sources of data to the system 100.

[0057] The components 105 to 113 of the computer module 101 typically communicate via an interconnected bus 104 and in a manner that results in a conventional mode of operation of the computer system 100 known to those in the relevant art. For example, the processor 105 is

13621053_l

2017228700 15 Sep 2017 coupled to the system bus 104 using a connection 118. Likewise, the memory 106 and optical disk drive 112 are coupled to the system bus 104 by connections 119. Examples of computers on which the described arrangements can be practised include IBM-PC’s and compatibles, Sun Sparcstations, Apple Mac™ or like computer systems.

[0058] The methods of rendering a surface to a plurality of output pixels may be implemented using the computer system 100 wherein the processes of Figs. 12 and 14-16, to be described, may be implemented as one or more software application programs 133 executable within the computer system 100. In particular, the steps of the methods of Figs. 12 and 14-16 are effected by instructions 131 (see Fig. IB) in the software 133 that are carried out within the computer system 100. The software instructions 131 may be formed as one or more code modules, each for performing one or more particular tasks. The software may also be divided into two separate parts, in which a first part and the corresponding code modules performs the described methods and a second part and the corresponding code modules manage a user interface between the first part and the user.

[0059] The software may be stored in a computer readable medium, including the storage devices described below, for example. The software is loaded into the computer system 100 from the computer readable medium, and then executed by the computer system 100. A computer readable medium having such software or computer program recorded on the computer readable medium is a computer program product. The use of the computer program product in the computer system 100 preferably effects an advantageous apparatus for rendering a surface to a plurality of output pixels.

[0060] The software 133 is typically stored in the HDD 110 or the memory 106. The software is loaded into the computer system 100 from a computer readable medium, and executed by the computer system 100. Thus, for example, the software 133 may be stored on an optically readable disk storage medium (e.g., CD-ROM) 125 that is read by the optical disk drive 112. A computer readable medium having such software or computer program recorded on it is a computer program product. The use of the computer program product in the computer system 100 preferably effects an apparatus for rendering a surface to a plurality of output pixels.

[0061] In some instances, the application programs 133 may be supplied to the user encoded on one or more CD-ROMs 125 and read via the corresponding drive 112, or alternatively may be read by the user from the networks 120 or 122. Still further, the software can also be loaded

13621053_l

2017228700 15 Sep 2017 into the computer system 100 from other computer readable media. Computer readable storage media refers to any non-transitory tangible storage medium that provides recorded instructions and/or data to the computer system 100 for execution and/or processing. Examples of such storage media include floppy disks, magnetic tape, CD-ROM, DVD, Blu-ray™ Disc, a hard disk drive, a ROM or integrated circuit, USB memory, a magneto-optical disk, or a computer readable card such as a PCMCIA card and the like, whether or not such devices are internal or external of the computer module 101. Examples of transitory or non-tangible computer readable transmission media that may also participate in the provision of software, application programs, instructions and/or data to the computer module 101 include radio or infra-red transmission channels as well as a network connection to another computer or networked device, and the Internet or Intranets including e-mail transmissions and information recorded on Websites and the like.

[0062] The second part of the application programs 133 and the corresponding code modules mentioned above may be executed to implement one or more graphical user interfaces (GUIs) to be rendered or otherwise represented upon the display 114. For example, the display 114 can be used to provide a GUI for the user to view and alter the surface of a product composed of a material. Through manipulation of the inputs 113, typically the keyboard 102 and the mouse 103, a user of the computer system 100 and the application may manipulate the interface in a functionally adaptable manner to provide controlling commands and/or input to the applications associated with the GUI(s). For example, the user can adjust inputs of the GUI such as sliders representing material appearance characteristics (e.g., sliders for colour reflectivity and the like) of the material. The application 133 executes the methods described below to render the surface including the adjusted material appearance. Other forms of functionally adaptable user interfaces may also be implemented, such as an audio interface utilizing speech prompts output via the loudspeakers 117 and user voice commands input via the microphone 180.

[0063] Fig. IB is a detailed schematic block diagram of the processor 105 and a “memory” 134. The memory 134 represents a logical aggregation of all the memory modules (including the HDD 109 and semiconductor memory 106) that can be accessed by the computer module 101 in Fig. 1A.

[0064] When the computer module 101 is initially powered up, a power-on self-test (POST) program 150 executes. The POST program 150 is typically stored in a ROM 149 of the

13621053_l

2017228700 15 Sep 2017 semiconductor memory 106 of Fig. 1A. A hardware device such as the ROM 149 storing software is sometimes referred to as firmware. The POST program 150 examines hardware within the computer module 101 to ensure proper functioning and typically checks the processor 105, the memory 134 (109, 106), and a basic input-output systems software (BIOS) module 151, also typically stored in the ROM 149, for correct operation. Once the POST program 150 has run successfully, the BIOS 151 activates the hard disk drive 110 of Fig. 1A. Activation of the hard disk drive 110 causes a bootstrap loader program 152 that is resident on the hard disk drive 110 to execute via the processor 105. This loads an operating system 153 into the RAM memory 106, upon which the operating system 153 commences operation. The operating system 153 is a system level application, executable by the processor 105, to fulfil various high level functions, including processor management, memory management, device management, storage management, software application interface, and generic user interface.

[0065] The operating system 153 manages the memory 134 (109, 106) to ensure that each process or application running on the computer module 101 has sufficient memory in which to execute without colliding with memory allocated to another process. Furthermore, the different types of memory available in the system 100 of Fig. 1A must be used properly so that each process can run effectively. Accordingly, the aggregated memory 134 is not intended to illustrate how particular segments of memory are allocated (unless otherwise stated), but rather to provide a general view of the memory accessible by the computer system 100 and how such is used.

[0066] As shown in Fig. IB, the processor 105 includes a number of functional modules including a control unit 139, an arithmetic logic unit (ALU) 140, and a local or internal memory 148, sometimes called a cache memory. The cache memory 148 typically includes a number of storage registers 144 - 146 in a register section. One or more internal busses 141 functionally interconnect these functional modules. The processor 105 typically also has one or more interfaces 142 for communicating with external devices via the system bus 104, using a connection 118. The memory 134 is coupled to the bus 104 using a connection 119.

[0067] The application program 133 includes a sequence of instructions 131 that may include conditional branch and loop instructions. The program 133 may also include data 132 which is used in execution of the program 133. The instructions 131 and the data 132 are stored in memory locations 128, 129, 130 and 135, 136, 137, respectively. Depending upon the relative size of the instructions 131 and the memory locations 128-130, a particular instruction may be

13621053_l

2017228700 15 Sep 2017 stored in a single memory location as depicted by the instruction shown in the memory location 130. Alternately, an instruction may be segmented into a number of parts each of which is stored in a separate memory location, as depicted by the instruction segments shown in the memory locations 128 and 129.

[0068] In general, the processor 105 is given a set of instructions which are executed therein. The processor 105 waits for a subsequent input, to which the processor 105 reacts to by executing another set of instructions. Each input may be provided from one or more of a number of sources, including data generated by one or more of the input devices 102, 103, data received from an external source across one of the networks 120, 102, data retrieved from one of the storage devices 106, 109 or data retrieved from a storage medium 125 inserted into the corresponding reader 112, all depicted in Fig. 1 A. The execution of a set of the instructions may in some cases result in output of data. Execution may also involve storing data or variables to the memory 134.

[0069] The described arrangements use input variables 154, which are stored in the memory 134 in corresponding memory locations 155, 156, 157. The described arrangements produce output variables 161, which are stored in the memory 134 in corresponding memory locations 162, 163, 164. Intermediate variables 158 may be stored in memory locations 159, 160, 166 and 167.

[0070] Referring to the processor 105 of Fig. IB, the registers 144, 145, 146, the arithmetic logic unit (ALU) 140, and the control unit 139 work together to perform sequences of microoperations needed to perform “fetch, decode, and execute” cycles for every instruction in the instruction set making up the program 133. Each fetch, decode, and execute cycle comprises:

a fetch operation, which fetches or reads an instruction 131 from a memory location 128, 129, 130;

a decode operation in which the control unit 139 determines which instruction has been fetched; and an execute operation in which the control unit 139 and/or the ALU 140 execute the instruction.

13621053_l

2017228700 15 Sep 2017 [0071] Thereafter, a further fetch, decode, and execute cycle for the next instruction may be executed. Similarly, a store cycle may be performed by which the control unit 139 stores or writes a value to a memory location 132.

[0072] Each step or sub-process in the processes of Figs. 12 and 14-16 is associated with one or more segments of the program 133 and is performed by the register section 144, 145, 147, the ALU 140, and the control unit 139 in the processor 105 working together to perform the fetch, decode, and execute cycles for every instruction in the instruction set for the noted segments of the program 133.

[0073] The methods described may alternatively be implemented in dedicated hardware such as one or more integrated circuits performing the functions or sub functions of the methods. Such dedicated hardware may include graphic processors, digital signal processors, or one or more microprocessors and associated memories.

[0074] When rendering a surface geometry that exhibits a particular material appearance, the shape of the input surface geometry is defined at a macro-level, which excludes any microstructures. In addition, the appearance of the material is also defined. The surface geometry and material appearance are used together in a rendering process to produce an output pixel buffer which contains a visual appearance of the surface.

[0075] The macro-level shape of the input surface geometry can be defined using a 3D mesh. The mesh represents a description of the surface, for example the surface of a product, to be rendered. Fig. 4 illustrates an example 3D mesh 410 used to define a macro-level shape, where normals to an input surface geometry, such as normal 420, have been defined for each vertex, for example a vertex 430. The normals (such as 420) can be referred to as local surface normals, and are interpolated across each face of the mesh 410 for locations that do not lie exactly on a vertex. Interpolation is achieved using well-established techniques such as bilinear interpolation.

[0076] The material appearance of the surface can be defined using a bi-directional reflectance distribution function (BRDF). Fig. 5 illustrates an example material whose appearance is characterised by a BRDF. Incoming coloured light 510 shines onto a location on a surface 520 of the material, with a local surface normal 530. The BRDF defines how the incoming coloured light will be reflected off the location on the surface 520, and therefore how much coloured light will be reflected back towards the viewing direction. Reflected light may not be the same

13621053_l

2017228700 15 Sep 2017 colour as the incoming light 510, depending on the properties of the material. BRDF implementations can divide the reflected light into a diffuse component 540 reflecting a low level of light equally in all directions, and a specular component 550 reflecting a larger amount of light more strongly in a specific direction. Viewers from some view directions, e.g. 560, 570 and 590, will observe predominantly the diffuse component 540, while other view directions, e.g. 580, will observe the diffuse component 540 in addition to the specular component 550.

[0077] The arrangements described relate to a method of rendering a surface into a plurality of output pixels. Fig. 12 shows an outline method 1200 of rendering a surface to a plurality of output pixels. The method 1200 is typically implemented as one or more modules of the application 133, stored in the memory 106 and controlled under execution of the processor 105.

[0078] The method 1200 begins at a receiving step 1210. A description of a surface to be rendered is received in execution of the step 1210 and stored in memory such as the memory 106. The description of the surface can be received in a form of three-dimensional mesh and represents a macro-level shape. The three-dimensional mesh may comprise a plurality of vertices connected in a polygonal manner with additional information associated with each vertex and/or polygon, as described in relation to Fig. 4. The additional information may include information describing material appearance, such as normal, bi-directional reflectance distribution function (BRDF) parameters, texture coordinates and the like. Alternatively the description may be in a form of a point cloud, with polygons formed from the point cloud. Alternatively, the step 1210 can relate to accessing a description of the surface, for example a description stored in the memory 106 or received from an external device via the network 120. The surface relates to a product or surface to be rendered in the GUI 114 and to be created using the material, for example using 3D printing.

[0079] The method 1200 proceeds under execution of the processor 105 from step 1210 to a receiving step 1220. A micro-structure description of variations of material appearance on the surface within each pixel of the set of output pixels to be rendered is received at step 1220. The micro-structure description defines micro-structures on the surface of the product (represented by the surface description received at 1210) to be rendered and at least a primary material and a secondary material for each of the micro-structures. Each material can be defined by a corresponding material appearance, such as apparent colours, roughness, anisotropy, reflectivity and the like, encoded, for example, in a bi-directional reflectance distribution function (BRDF). The micro-structure description relates to a statistical model of orientation and description of

13621053_l

2017228700 15 Sep 2017 micro-structure shapes. The micro-structure description is associated with a plurality of representative micro-structure faces having at least a primary material appearance and a secondary material appearance, different from the primary material appearance. For example the primary material appearance can relate to the perceived appearance 320 of Fig. 3 and the secondary material appearance to the perceived appearance 330 of Fig. 3. The micro-structure description may be in a form of a plurality of normals for a representative micro-structure geometry such as a pyramid. The micro-structure description can also include areas for faces of the representative pyramid, for example as described in relation to Fig. 3. Alternatively, the micro-structure description may include vertices of a representative pyramid or an arrangement of pyramids. The primary material appearance and the secondary material appearance relate to variations in a parameter of a BRDF function, for example, relating to different colours, roughness, anisotropy, reflectivity parameters and the like. The step 1220 can also relate to accessing a micro-structure appearance stored in the memory 106, or receiving adjustment to a material appearance by user interaction with the input 113 and the GUI display 114 of the computer module 101.

[0080] The method 1200 continues under execution of the processor 105 from step 1220 to a determining step 1230. At execution of step 1230, a visibility measure for each visible representative face is determined based on a normal of the surface at a position of a particular output pixel, a viewing direction and the micro-structure description. The normal of the surface is determined from the description of the surface received at step 1210. The visibility measure is determined by, for example, determining a relative visibility of each representative face using a dot product of the representative micro-structure face normal with the viewing direction for each representative micro-structure face normal. Positive dot products are facing the viewer, and all other dot products are discarded. The positive dot products are normalised by dividing each dot products by the sum of the positive dot products for all representative faces for a particular pixel. The resulting normalised dot products are used as the visibility measure for their corresponding representative micro-structure face normal. The step 1230 is executed for each pixel in the plurality of output pixels. A method 1500 of determining a visibility measure for an output pixel, as implemented at step 1230 is described below in relation to Fig. 15.

[0081] The method 1200 continues under execution of the processor 105 from step 1230 to a determining step 1240. At step 1240, the visibility measures of the visible representative microstructure faces determined at step 1230 are used to determine an aggregate material appearance for the output pixel. The aggregate material appearance is determined by combining the primary

13621053_l

2017228700 15 Sep 2017 material appearance and the secondary material appearance weighted based on the determined visibility measures of the visible representative micro-structure faces. The contribution of each of the primary and secondary material appearances is based on the weighting. The step 1230 is implemented for each output pixel.

[0082] The method 1200 continues under execution of the processor 105 from step 1240 to a determining step 1250. The output pixel is rendered in execution of the step 1250 using the aggregate material appearance and output into a output pixel buffer for displaying on the display 114. The step 1250 is executed for each pixel in the plurality of output pixels. A method 1400 of rendering an output pixel, relating to implementation of steps 1240 and 1250, is described below in relation to Fig 14 [0083] In one arrangement, an input surface geometry which exhibits a colour travel material is rendered to an output pixel buffer. The corresponding method 1400 of rendering an input surface geometry to an output pixel buffer, as implemented at steps 1240 and 1250 is shown in Fig. 14. The method 1400 is typically implemented as one or more modules of the application 133, stored in the memory 106 and controlled under execution of the processor 105.

[0084] The method 1400 begins at step 1410. In execution of the step 1410, the input surface geometry is determined. The method 1400 then proceeds under control of the processor 105 to a determining step 1420. In execution of step 1420, the material appearance information of the surface geometry is determined. The material appearance relates to the aggregate material appearance for an output pixel. The method 1400 then proceeds to a rendering step 1430. In execution of the step 1430, the surface geometry is rendered into an output pixel buffer, using the material appearance information. The method 1400 then proceeds to a display step 1440. In execution of the step 1440 the output pixel buffer is displayed, for example via a GUI reproduced on the display 114. After step 1440, the method 1400 ends.

[0085] The colour travel material is attempting to simulate a physically fabricated colour travel material which is to be covered in micro-structures, such as square-based pyramids. Fig. 6 shows an example square-based pyramid 610. The pyramid 610 is oriented around a local surface normal 620, with micro-face normals 630 640 assigned to each face. Only normals for visible faces are shown in Fig. 6. The base of the pyramid 610 is not considered, as the base is not intended for the modelled surface to be rendered from that direction.

13621053_l

2017228700 15 Sep 2017 [0086] The geometry of the micro-structure can also be parameterised. In one arrangement, there is one parameter, as shown for a pyramid 700 in Fig. 7. An angle of sides 710 of the pyramid 700 is determined by a parameter which describes a height 720 of the pyramid relative to a width of the pyramid 700. Different values of the parameter model different ways that the pyramid 700 can be fabricated.

[0087] Each face of the pyramid 700 is also associated with a BRDF. The BRDF for each face describes the material appearance of that face, as described above. Each face can be associated with a different BRDF, or some faces can be associated with the same BRDF. In the arrangements described, opposing faces have the same BRDF and adjacent faces have different BRDFs.

[0088] The micro-structure is also associated with a visibility function. A visibility function 1320, described in Fig. 13, determines the visibility of the micro-structure faces, as a function of the viewing direction. The function 1320 receives an input and returns an output. As input, the function 1320 receives a micro-structure description 1310 transformed using the local surface normal and the viewing direction. The micro structure description 1310 includes representative micro-structure faces, or some other means of describing the geometry of the micro-structure. It is assumed that the micro-structure faces have already been transformed by any parameterisation. As output, the function 1320 returns a visibility weighting for each visible micro-structure face as a proportion of all representative micro-structure faces 1330 that are visible from the viewing direction. The visibility weighting can be also considered as a relative contribution of the corresponding visible micro-structure face compared to contribution of other visible faces to overall appearance of the output pixel. The relative contribution of a visible face is determined based on the viewing direction, the local surface normal and a face normal defined by the micro-structure description. As such, the function 1320 returns the relative contribution of each visible micro-structure face to overall appearance of the output pixel in the form of visibility weighting and accordingly the contribution of the material appearance associated with each face, to each output pixel in the output pixel buffer.

[0089] Figs. 8a to 8c illustrate some examples of executing the visibility function 1320 under different circumstances. In Fig. 8a, a micro-structure pyramid 810 is at a 45 degree angle to a viewing direction. Two micro-structure faces, 820 and 830, are visible, each of the faces 820 and 830 with a corresponding normal 825 and 835. The results of executing the visibility function 1320 are that two faces are equally visible, one face (830) at 50% of total pyramid

13621053_l

2017228700 15 Sep 2017 visibility with appearance 850, and the other face (820) also at 50% of total pyramid visibility with appearance 855.

[0090] In Fig. 8b, the viewing angle is rotated, showing a micro-structure pyramid 810 at a different angle to the angle of the structure 810 in Fig. 8a. The same two micro-structure faces 820 830 are still visible, each with the corresponding normals 825 835. The output of the visibility function is that two faces are still visible, but they are no longer equally visible. One face (830) is at 70% of total pyramid visibility 860, and the other face (820) is only at 30% of total pyramid visibility 865.

[0091] In Fig. 8c, the viewing angle is further rotated, showing the micro-structure pyramid 810 such that the micro-structure face 830 is oriented predominantly towards the viewing direction. Three micro-structure faces 820, 830 and 840 are visible to the viewer, each face with a corresponding normal 825, 835 and 845. The results of the visibility function 1320 show that three faces are visible. One micro-structure face 830 is at 90% of total pyramid visibility 870, a second micro-structure face 820 is at 5% of total pyramid visibility875, and a third microstructure face 840 is also at 5% of total pyramid visibility 880. Two visible micro-structure faces 820 840 have the same BRDF, but have different normals 825 845. Therefore the faces 820 and 840 are treated separately by the visibility function 1320, producing two visibility outputs 875 880.

[0092] In the arrangement described, the visibility function 1320 assumes that the microstructure description consists only of the normals of the micro-structure faces, the normals being assumed to be unit-length. The relative visibility weightings of each face are proportional to the dot product of the micro-structure face normal with the unit-length view direction. The unit-length view direction is assumed to be pointing away from the surface. Negative dot products, representing micro-structure faces that are not oriented towards the viewing direction, are ignored.

[0093] Related to the arrangement using the unit-length view direction, the visibility function

1320 assumes that the micro-structure description also includes a relative area of the representative micro-structure face. The relative area is determined with respect to other faces of the micro-structure description. For example, a relative area for each face of a pyramid is determined as a contribution of that face to the total surface area of the pyramid viewed by the

13621053_l

2017228700 15 Sep 2017 operator. The dot product is weighted by the relative area to produce the visibility weighting for that micro-structure face.

[0094] All visibility weightings for all visible representative micro-structure faces are normalised, such that the resulting relative visibility weightings sum to 100% of total pyramid visibility.

[0095] Fig. 15 shows the method 1500 for determining the relative visibility weightings (visibility measure) using the normals of the micro-structure faces, as implemented at step 1230 of the method 1200. The method 1500 is typically implemented as one or more modules of the application 133, stored in the memory 106 and executed under control of the processor 105. The method 1500 relates to an example implementation of the visibility function 1320.

[0096] The method 1500 begins at step 1510. In execution of step 1510, a first micro-structure face is set as the current micro-structure face. The method 1500 then proceeds to step 1520. In execution of step 1520, a dot product of the current micro-structure face normal and the unitlength viewing direction vector is determined. The method 1500 then proceeds to step 1530. In execution of step 1530, the dot product is examined to determine whether the dot product is negative. Steps 1520- 1560 form a loop which iterates over every micro-structure face normal.

[0097] If the dot product is negative (“Yes” at step 1530), the method 1500 proceeds to step 1540. In execution of step 1540, the next micro-structure face is set as the current micro structure face. Steps 1530, 1540 and 1520 form a loop which iterates over every micro-structure face normal having a negative dot product.

[0098] If the dot product is not negative (“No” at step 1530), the method 1500 proceeds to step 1550. The dot product is stored, and associated with the current micro-structure face at step 1550.

[0099] The method 1500 proceeds to step 1560. In step 1560, the application 133 determines if there are any further micro-structure faces yet to be processed.

[00100] If there are further micro-structure faces yet to be processed (“Yes” at step 1560), the method 1500 proceeds to step 1540, as previously described. Steps 1560, 1540 and 1520 form a

13621053_l

2017228700 15 Sep 2017 loop that iterates over every micro-structure face normal which does not face away from the viewing direction.

[00101] If there are no further micro-structure faces yet to be processed (“No” at step 1570), the method 1500 proceeds to step 1570. At step 1570, the visibility weighting (also referred to as a visibility measure) for each visible micro-structure face is determined from the stored dot product associated with each respective micro-structure face.

[00102] The method 1500 then proceeds to step 1580. In execution of step 1580, the visibility weightings are normalised, such that all visibility weightings sum to 100% of total pyramid visibility. For example, a visibility weighting for a visible face of the micro-structure description can be determined as a corresponding normalised dot product. The dot product can be normalised by dividing the dot product by the sum of dot products associated with all visible faces of the micro-structure description.

[00103] After step 1580, the method 1500 ends. The method 1500 is particularly suitable for simple micro-structures with minimal self-occlusion, such as square pyramids or truncated square pyramids.

[00104] More micro-structure shapes can be employed using the arrangements described by associating each micro-structure face with a value that describes the area of the micro-structure face relative to other micro-structure faces. The visibility weighting for each micro-structure face is multiplied by the relative area value for the micro-structure face, prior to the normalisation step 1580.

[00105] In another arrangement, the visibility function 1320 assumes that the micro-structure description includes only the vertices of the representative micro-structure faces. A sufficiently large array of micro-structure instances is constructed for each output pixel with a unique colour assigned to all instances of each representative micro-structure face. The array is considered to be sufficiently large if the array completely fills a render viewport for off-screen rendering. The render viewport is determined based on boundaries of the output pixel to be rendered. The array for each output pixel is rendered into an associated off-screen pixel buffer using the viewing direction (angle) to determine the render viewpoint. The viewpoint is assumed to be at an infinite distance from the surface, so an orthogonal projection is used that produces no perspective distortion. The orthogonal projection is defined to fit at least one

13621053_l

2017228700 15 Sep 2017 complete micro-structure instance within the viewport boundaries, although more than one complete micro-structure is preferred. The render is performed with false lighting to light each face equally, and without any form of anti-aliasing or shadowing, so that only the unique face colours are rendered into the off-screen pixel buffer. The off-screen pixel buffer is examined to identify pixels corresponding to each of the unique face colours that were rendered. The number of pixels for each particular micro-structure face colour is determined by summing the corresponding identified pixels. The sums of corresponding identified pixels are normalised, and the normalised sums are returned as the relative visibility of the micro-structure faces as a percentage of total micro-structure visibility. The arrangement assuming that the microstructure description includes only the vertices of the representative micro-structure faces is most suitable for complex micro-structures where self-occlusion is a notable feature of the micro-structure. Accordingly, the step 1230 is performed during run-time (as the user inputs adjustments of material appearance via the inputs 113) by performing an off-screen render of the micro-structures at the viewing angle and counting the proportion of the micro-structure faces that are visible.

[00106] In a further arrangement, the off-screen renders of the microstructures of the previous arrangement are performed off-line for every representable viewing direction. Representable viewing directions are determined based on precision of the application 133 in determining and implementing the directions. In the further arrangement, the step 1230 relates to execution of off-screen renders of the micro-structures before run-time at all representable viewing angles to determine corresponding visibility measures. The resulting visibilities are stored in a look-up table, keyed by viewing direction. At run-time, the visibility function 1320 assumes that the micro-structure description includes only the table of stored visibilities, and performs a look-up into the table, using the viewing direction as the search key. The visibilities found in the table are returned from the visibility function as the relative visibility of the micro-structure faces as a percentage of total micro-structure visibility. The arrangement performed off-line for every representable viewing direction is particularly suitable for complex micro-structures where a large number of faces precludes the use of the secondary arrangement in real-time. Having a large number of faces is particularly useful for approximating micro-structures with curved surfaces.

[00107] In a final arrangement, the determination of the visibility measure is performed for a subset of every representable viewing direction, and the resulting visibilities are stored in a look-up table, keyed by viewing direction. Accordingly, step 1230 relates to performing off13621053_l

2017228700 15 Sep 2017 screen renders on a subset of the representable viewing angles. Determining the visibility measure relates interpolating values in the look-up table. At run-time, the visibility function assumes that the micro-structure description includes only the table of stored visibilities, and performs a look-up into the table, using the viewing direction as the search key. If an exact match for the viewing direction is not found in the table, multiple intermediate visibilities are returned for nearby viewing directions, and the final visibilities are determined by interpolating the intermediate visibilities. The final visibilities are returned from the visibility function as the relative visibility of the micro-structure faces as a percentage of total micro-structure visibility. The number and configuration of the subset of every viewing direction can be chosen as required to achieve the desired balance between memory usage of the table and accuracy of the final visibilities. The method of interpolation can also be chosen as required to achieve the desired balance between computational cost and accuracy of the final visibilities. Accordingly, the step 1230 is performed partially off-line prior to run-time and partially (using interpolation) at run-time. The arrangement performed for a subset of representable viewing directions is particularly suitable for complex micro-structures where a large number of faces precludes the use of the secondary arrangement in real-time, and memory constraints preclude the use of the complete table from the tertiary arrangement.

[00108] Storing pre-determined visibilities in a table is typically more efficient that storing prerendered BRDF results directly in a table. Storing visibilities requires a smaller table than storing BRDF results, since the visibility is a function of a single variable (viewing direction) and is independent of sources of light. In contrast, the BRDF result is a function of at least three variables (viewing direction, light direction, light intensity) and is strongly dependent on sources of light in the scene. In addition, storing a subset of visibilities is less sensitive to interpolation than storing a subset of BRDF results, especially for materials which are highly reflective and glossy.

[00109] Regardless of the particular arrangement, the visibility function determines the relative visibility of the micro-structure faces for a given viewing direction, as a percentage of total micro-structure visibility.

[00110] When rendering a material with micro-structures, the micro-structures are assumed to be smaller than an individual pixel. The rendering procedure considers every pixel independently. A rendering method 1600 is described in Fig. 16. The method 1600 is typically implemented as one or more modules of the application 133, stored in the memory 106 and

13621053_l

2017228700 15 Sep 2017 executed under control of the processor 105. The method 1600 provides an alternative arrangement to steps 1230 to 1250 of the method 1200.

[00111] The method 1600 assumes that the local surface normal is known for the location on the surface geometry that intersects a ray cast from the camera location through the current pixel, along with the view direction, the micro-structure faces, any micro-structure parameterisation, the visibility function, and the assignment of BRDFs to micro-faces. The output of the rendering method 1600 is a single rendered output pixel.

[00112] The method 1600 begins at step 1610. At step 1610, the micro-structure face geometry is determined according to the micro-structure parameterisation, if any. The method 1600 proceeds to step 1620. In execution of the step 1620, the angles of the micro-structure faces are determined relative to the view direction. The method 1600 then proceeds to step 1630. In execution of step 1630, the visibility function is used to determine the visibility weighting of each visible micro-structure face, according to method 1500. The method 1600 proceeds to step 1640. In execution of step 1640, the visible micro-structure faces are rendered into a temporary off-screen pixel buffer according to the associated BRDFs of the faces. The method 1600 proceeds to step 1650. In execution of step 1650, the temporary off-screen pixel buffers are blended according to the visibility weightings of the corresponding visible micro-structure faces, producing the output rendered pixel. After step 1650, the method 1600 ends.

[00113] Step 1640 of rendering according to the associated BRDF may be performed by directly determining the value of the BRDF, given a light source direction, the normal direction of the micro-structure face, and the viewing angle, according to the mathematical definition of the BRDF function. The approach of rendering according to the associated BRDF is suitable for a point source light. However, a pure point source light may give degenerate or unrealistic looking results for highly glossy materials. An alternative is to use a more physically plausible area light source while still employing a point source light rendering methods. For a sufficiently small area, the difference in visibility and angle for different light source angles within the area light source may be ignored. However, rendering according to the associated BRDF does require calculating the integral of the BRDF over the area of the light source.

[00114] In some cases, the integral of the BRDF may be too complex to be determined in real time. One known solution is to use a BRDF expressed as a Linearly Transformed Cosine (LCT), as efficient methods of performing an integral over a polygonal area are known, as described by

13621053_l

2017228700 15 Sep 2017

Heitz et al (Eric Heitz; Jonathan Dupuy; Stephen Hill; David Neubelt, “Real-Time PolygonalLight Shading with Linearly Transformed Cosines”, ACM Transactions on Graphics, July 2016) for example. If the light source shape and BRDF form are known in advance, such as in part of a system rendering different materials in a controlled environment, another alternative arrangement is to determine the integral of the BRDF (or of a parameterised family of BRDFs) with the known light source geometry in advance, and render the pre-integrated BRDF. Mathematically, the integral of a BRDF with a particular area light source is equivalent to using a different BRDF, as in either case the result is a function taking light and viewing angles (relative to a surface normal) as input and giving a proportion of reflected light as output.

[00115] The integral of an arbitrary BRDF with a particular light source shape may not have a known analytic solution, or the analytic solution may be impractically complex. In this case, the integral may be approximated numerically. The approximated integral may be represented as a look-up table, or may be further approximated with a suitable functional form. One useful functional form for a typical BRDF and a uniform spherical light source (a light source with a circular cross section from any direction) is the exponential power distribution, which takes the following form:

A e a Equation (1) [00116] Equation (1) is expressed in terms of the half-angle Θ, being is the angle between the half-vector and the surface normal, where the half vector is the average of the light source and viewing direction vectors. Equation (1) raises the angle θ to the power of a parameter β, then divides by a parameter a, and raises e to the power of the resulting number. The result is multiplied by a normalisation term A. The normalisation term A depends on the parameters a and β and ensures that the integral with respect to θ over the whole hemisphere is equal to 1. With an appropriate choice of parameters a, and β, the function of Equation (1) may be used to approximate the integral of a typical (family of) anisotropic BRDF(s) with a spherical light source Equation (1) may be used to approximate the integral because combinations of the two parameters can be used to control both the width and the falloff of the peak of the function, which matches the range of shapes taken by the integral of a spherical light source with a typical BRDF.

13621053_l

2017228700 15 Sep 2017 [00117] As such, a reflectivity distribution representing reflectivity of at least one material under illumination of an area-based light source, for example a spherical light source, may be determined using properties (radius) of the area-based light source and the roughness of the material. The method of determining the reflectivity distribution effectively simulates illumination of the surface by an area-based light source by modifying the reflectivity distribution to represent reflectivity of the material under illumination of the area-based light source. The reflectivity distribution is modified by deriving a distribution with a separate or independent control of a steepness and a width of the derived distribution. The steepness of the derived distribution is based on the roughness of the material and the width of the derived distribution is based on a size of the light source.

[00118] The arrangements described allow rendering of the surface under illumination of a spherical light source in accordance with a point light source scene rendering method generally known in the art by using the determined reflectivity distribution. As such, brightness of pixels corresponding to the material is determined using the determined reflectivity distribution to simulate illumination of the material by the area-based light source.

[00119] The arrangements described offer several advantages. Firstly, the arrangements described can provide a greater accuracy than simple colour-to-colour renderers, as the arrangements described incorporate into the rendering model the physical structure that provides the colour travel behaviour. A visible consequence of the greater accuracy is the introduction of multiple specular highlights for a single light source on a nominally smooth input surface geometry. Since the surface uses a model that simulates multiple micro-structures with faces at different angles, the faces all reflect light at a different angle to the nominal face of the input surface geometry. Secondly, the arrangements described do not require significant computation resources, making operation of the described methods efficient enough to operate interactively in real-time, e.g. at 30-60 frames per second on typical mid-range graphics hardware.

[00120] In the context of the arrangements described, a microstructure relates to a set of flat faces joined so as to form a three-dimensional object. For example, the micro-structures can be truncated square pyramids, rather than square pyramids. Fig. 9 shows a truncated square pyramid 910, oriented around a local surface normal 920. The pyramid 910 has micro-face normals 930, 940 and 950 assigned to each face. Only normals for visible faces are shown in Fig. 9. As with the square pyramid 610, the base of the truncated square pyramid 910 is not

13621053_l

2017228700 15 Sep 2017 considered in the arrangements described, as the micro-structure is not intended to be viewed from the base direction.

[00121] The geometry of the truncated square pyramid micro-structure 910 is also parameterised. In one embodiment, there are two parameters, as shown in Fig. 10 for a truncated pyramid 1010. The angle of the sides of the truncated square pyramid 1010 is determined by a parameter describing a height 1020 of the un-truncated square pyramid relative to the width of the pyramid 1010. A second parameter 1030 describes a relative height 1030 of the truncation above the base of the pyramid 1010.

[00122] As with square pyramid micro-structures, each face of the pyramid 1010 is also associated with a BRDF. Each face can be associated with a different BRDF. However in the example of Fig. 10, one set of opposing faces share the same BRDF as the top face, while the remaining opposing faces share a different BRDF.

[00123] In the example of Figs. 9 and 10, the remaining features, such as the visibility functions and rendering procedure, operate in principle the same way as described in relation to Fig. 14, Fig. 13, Fig. 8, and Fig. 16.

[00124] The arrangements described can be applied where the micro-structures relate to different shapes, for example, shapes with more or less sides than a square pyramid, inverted shapes, truncated shapes, shapes with self-occlusion, and shapes with overhangs or holes. A large number of shapes are possible, provided the shapes are comprised of a finite number of flat faces. Fig. 11 shows a collection of three example micro-structures: a truncated hexagonal pyramid 1110, an inverted concave truncated square pyramid 1120, and a more complex shape which exhibits self-occlusion 1130. Curved micro-structure sides can be approximated with a large number of flat faces.

[00125] Further embodiments consider the micro-structure parameters to vary spatially over the surface geometry, but being constant within a single pixel. For each parameter in the microstructure parameterisation that is to vary spatially, the values of the spatially varying parameter can be stored in a texture which maps onto the surface geometry. For example, a microstructure shape (such as a pyramid) associated with one output pixel can differ from a microstructure shape (such as a truncated pyramid) associated with another output pixel.

13621053_l

2017228700 15 Sep 2017 [00126] Further, the arrangements described can relate to embodiments where the microstructures are spatially distributed according to a micro-structure distribution value. The spatial distribution may be random or non-random. Gaps or spaces between micro-structures become possible, and are handled with the definition of an additional BRDF representing the material between the micro-structures. The visibility function determines the contribution of the gap BRDF to the final visibilities of the micro-structure face BRDFs using the micro-structure distribution value. The micro-structure distribution value can be constant, or can vary spatially over the surface geometry.

[00127] Yet further additional embodiments can be applied to micro-structures randomly rotated about the local surface normal according to a rotation distribution. Typically, the rotation distribution will be defined statistically, rather than discretely. The rotation distribution causes each micro-structure face normal to become a distribution of rotated micro-structure face normals for each micro-structure face. A single normal, such as a micro-structure face normal, describes a flat face. However a distribution of normals, such as a distribution of rotated microstructure face normals, can describe a curved surface. Accordingly, micro-structures rotated according to a rotation distribution providing a distribution of rotated micro-structure face normals for each face can be used to approximate micro-structures with curved surfaces. The original BRDF associated with each micro-structure face is then modified from describing the reflection characteristic with the micro-structure face normal, to describing the reflection characteristics over a distribution of normals, given by the micro-structure face normal modified by the rotation distribution. The original visibility function associated with the microstructure is also modified to take the rotation distribution into account.

[00128] For some classes of rotation distributions, such as uniform distributions, the modified BRDF and modified visibility function can be approximated by convolving the BRDF and, optionally, the visibility function with the rotation distribution. The convolution integrates the BRDF responses over the normals represented by the rotation distribution. Convolving separately over the BRDF and visibility functions can introduce errors at viewing angles where the visibility function is low (e.g. viewing angles close to orthogonal to the micro-structure normal). The error is reduced when the range of angles in the rotation distribution is relatively small.

[00129] Depending on the BRDF and rotation distribution, the resulting convolution may not have a simple analytic solution. If the convolution does not have a simple analytic solution, the

13621053_l

2017228700 15 Sep 2017 modified functions may be either approximated numerically, or further approximated using a suitable functional form: for example, the exponential power distribution, as described for approximating an area light source. For other classes of rotation distributions which are more complicated, the modified BRDF and modified visibility function can be approximated by sampling the original BRDF at appropriate rotations and calculating a weighted average. In the case of the modified visibility function, the weighted average can be performed offline and then stored in a table, as described in previous embodiments.

[00130] The arrangements described are useful in a system for previewing a material appearance, e.g. for a material with a colour travel effect, intended for physical fabrication such as through 2.5D printing. An example system presents the user with a rendering of the colour travel material, for example on the display 114, along with a graphical user interface for adjusting various properties of the colour travel material. The properties of the colour travel material can include the shape of the surface, the shape of the micro-structures on the surface, the values of any micro-structures parameters, and the values of material appearance properties, such as colour, roughness, reflectivity, anisotropy, and the like. Users can review the rendering of the colour travel material before fabrication, and adjust any properties as needed. Review of the material with a colour travel effect allows the user to achieve the desired colour travel effect in the fabricated output without wasting time and resources by unknowingly fabricating an output with an undesired colour travel effect.

[00131] An example use of the arrangements described relates to a user of the device 101 developing a product with a desired surface appearance. Using the GUI displayed via the display 114, the user manipulates the inputs 113 to adjust properties of the material such as reflectivity or colour. The application 133 accordingly executes the method 1200 from steps 1220 to 1250 to use the micro-structure description with the adjusted appearance characteristic, determine the weighting for each output pixel, the aggregated material appearance for each output pixel and render the output pixels on the display 114 [00132] The foregoing describes only some embodiments of the present invention, and modifications and/or changes can be made thereto without departing from the scope and spirit of the invention, the embodiments being illustrative and not restrictive.

13621053_l

2017228700 15 Sep 2017 [00133] The arrangements described are applicable to the computer and data processing industries and particularly for the graphics and graphical material or product development industries.

[00134] In determining the weighting of each visible face per output pixel and generating an aggregate material appearance for each output pixel based on the weighting, the arrangements described generate the aggregate appearance based on physical properties of the material surface. The resultant appearance is more likely to be accurate than previously known methods that relate to mathematical models of desired properties of the material. Further, the arrangements described can be applied to an entire surface rather than relatively small portions of the surface.

[00135] Some known arrangements use physical properties of the material surface. However, the known methods typically explicitly model the micro-structure geometry in the mesh data, rather than determining the visibility measure (weighting) of the face for each pixel and generating an aggregate appearance. The resultant rendering in known methods requires orders of magnitude more mesh data to render than the arrangements described. The arrangements described are accordingly more computationally efficient and allow real-time updates to be displayed via the GUI to the user.

[00136] In the context of this specification, the word “comprising” means “including principally but not necessarily solely” or “having” or “including”, and not “consisting only of’. Variations of the word comprising, such as “comprise” and “comprises” have correspondingly varied meanings.

Claims (20)

1. A method of rendering a surface into a plurality of output pixels, the method comprising:

receiving a description of the surface;

for each of the plurality of output pixels, receiving a micro-structure description of variations of material appearance on the surface within said pixel, wherein the micro-structure description is associated with a plurality of representative micro-structure faces, each of the plurality of micro-structure faces being associated with one of at least a primary material appearance and a secondary material appearance;

determining, for each of the plurality of output pixels, a visibility measure for each visible representative micro-structure face based on a normal of the surface at the position of the pixel, a viewing direction and the corresponding micro-structure description, the normal of the surface being determined from the received description of the surface;

determining, for each of the plurality of output pixels, an aggregate material appearance for said pixel by combining the primary material appearance and the secondary material appearance, the contribution of each of the primary and secondary material appearances weighted based on the determined visibility measures of the visible representative microstructure faces; and rendering each of the output pixels using the determined aggregate material appearance to render the surface.

2. The method according to claim 1, wherein the micro-structure description comprises one of a) a set of normals for a micro-structure geometry, b) a set of vertices for the representative micro-structure geometry, c) a set of normals and faces for the micro-structure geometry, and d) a look-up table.

13621053_l

2017228700 15 Sep 2017

3. The method according to claim 1, wherein the primary material appearance relates to a primary perceived colour of the material and the secondary material appearance relates to a secondary perceived colour of the material.

4. The method according to claim 1, wherein the visibility measures are determined using normals of the micro-structure faces associated with the output pixel, the normals being determined from the micro-structure description and the normal of the surface.

5. The method according to claim 4, further comprising:

determining visible faces of the micro-structure faces using normals of the microstructure faces and the viewing direction;

determining a relative contribution of each micro-structure face to appearance of the output pixel compared to contribution of other visible faces using normals of the visible faces and the viewing direction.

6. The method according to claim 4, wherein the visibility measures are further based on a relative area of each micro-structure face.

7. The method according to claim 1, wherein the visibility measure is determined at runtime by performing an off-screen render of the micro-structures at the viewing angle and determining a relative contribution of each micro-structure face that is visible to appearance of the output pixel.

8. The method according to claim 1, wherein off-screen renders of the micro-structures are performed before run-time for a plurality of representable viewing angles to determine corresponding visibility measures, the visibility measures being stored in a look-up table.

9. The method according to claim 8, wherein determining the visibility measure relates to interpolating values in the look-up table.

10. The method according to claim 1, wherein the micro-structure relates to one or more of a pyramid, an inverted pyramid, truncated pyramid, cylinder, shapes with self-occlusion, and shapes with overhangs or holes.

13621053_l

2017228700 15 Sep 2017

11. The method according to claim 1, wherein a distribution of normals of the representative micro-structure faces describes a shape with at least one curved surface.

12. The method according to claim 1, wherein a distribution of rotated face normals for each representative micro-structure faces approximates a micro-structure shape with at least one curved surface.

13. The method according to claim 1, wherein the micro-structures are spatially distributed according to a micro-structure distribution value.

14. The method according to claim 13, wherein a bi-directional reflectance distribution function (BRDF) of the material is assigned to spaces between the micro-structures.

15. The method according to claim 1, wherein a micro-structure description associated with one of the output pixels differs from a micro-structure description associated with another of the output pixels.

16. The method according to claim 1, wherein the rendering comprises:

modifying a reflectance distribution of a material using properties of an area-based light source; and rendering the surface under illumination of the area-based light source using the modified reflectance distribution in accordance with a point light source rendering method.

17. The method according to claim 1, wherein the rendering comprises:

simulating illumination of the surface by an area-based light source by modifying a reflectivity distribution associated with a material to represent reflectivity of the material under illumination of the area-based light source, modifying the reflectivity distribution comprising deriving a distribution with a separate control of a steepness and a width of the distribution, wherein the steepness of the derived distribution is based on the roughness of the material and the width of the derived distribution is based on a size of the light source; and

13621053_l

2017228700 15 Sep 2017 rendering the surface in accordance with a point light source rendering method using the modified reflectivity distribution to generate a rendered output, wherein the rendered output simulates illumination of the material by the area-based light source.

18. A system comprising:

a memory; and a processor, wherein the processor is configured to execute code stored on the memory for implementing a method of rendering a surface into a plurality of output pixels, the method comprising:

receiving a description of the surface;

for each of the plurality of output pixels, receiving a micro-structure description of variations of material appearance on the surface within said pixel, wherein the micro-structure description is associated with a plurality of representative micro-structure faces, each of the plurality of micro-structure faces being associated with one of at least a primary material appearance and a secondary material appearance;

determining, for each of the plurality of output pixels, a visibility measure for each visible representative micro-structure face based on a normal of the surface at the position of the pixel, a viewing direction and the corresponding micro-structure description, the normal of the surface being determined from the received description of the surface;

determining, for each of the plurality of output pixels, an aggregate material appearance for said pixel by combining the primary material appearance and the secondary material appearance, the contribution of the each of the primary and secondary material appearances weighted based on the determined visibility measures of the visible representative microstructure faces; and rendering each of the output pixels using the determined aggregate material appearance to render the surface.

13621053_l

2017228700 15 Sep 2017

19. A non-transitory computer readable medium having a computer program stored thereon to implement method of rendering a surface into a plurality of output pixels, the method comprising:

receiving a description of the surface;

for each of the plurality of output pixels, receiving a micro-structure description of variations of material appearance on the surface within said pixel, wherein the micro-structure description is associated with a plurality of representative micro-structure faces, each of the plurality of micro-structure faces being associated with one of at least a primary material appearance and a secondary material appearance;

determining, for each of the plurality of output pixels, a visibility measure for each visible representative micro-structure face based on a normal of the surface at the position of the pixel, a viewing direction and the corresponding micro-structure description, the normal of the surface being determined from the received description of the surface;

determining, for each of the plurality of output pixels, an aggregate material appearance for said pixel by combining the primary material appearance and the secondary material appearance, the contribution of the each of the primary and secondary material appearances weighted based on the determined visibility measures of the visible representative microstructure faces; and rendering each of the output pixels using the determined aggregate material appearance to render the surface.

20. Apparatus configured to render a surface into a plurality of output pixels, comprising:

a memory;

a processor configured to execute code stored on the memory to:

receive a description of the surface;

for each of the plurality of output pixels, receive a micro-structure description of variations of material appearance on the surface within said pixel, wherein the micro-structure

13621053_l

2017228700 15 Sep 2017 description is associated with a plurality of representative micro-structure faces, each of the plurality of micro-structure faces being associated with one of at least a primary material appearance and a secondary material appearance;

determine, for each of the plurality of output pixels, a visibility measure for each visible representative micro-structure face based on a normal of the surface at the position of the pixel, a viewing direction and the corresponding micro-structure description, the normal of the surface being determined from the received description of the surface;

determine, for each of the plurality of output pixels, an aggregate material appearance for said pixel by combining the primary material appearance and the secondary material appearance, the contribution of the each of the primary and secondary material appearances weighted based on the determined visibility measures of the visible representative micro-structure faces; and render each of the output pixels using the determined aggregate material appearance to render the surface.

Canon Kabushiki Kaisha

Patent Attorneys for the Applicant

SPRUSON & FERGUSON

13621053_l

2017228700 15 Sep 2017

1/17

FIG. 1A

13621054 1

2017228700 15 Sep 2017

2/17

133

134

ROM 149 POST 150 BIOS 135

Bootstrap Loader 152

Input Variables 1354

I 132

Operating

System 153

Output Variables 161

Intermediate Variables 158

159 166 160 167

FIG. 1B

13621054 1

2017228700 15 Sep 2017

3/17

13621054 1

2017228700 15 Sep 2017

4/17

330

300

320

Fig. 3

13621054 1

420

430

2017228700 15 Sep 2017

5/17

Fig. 4

13621054 1

2017228700 15 Sep 2017

6/17

Fig. 5

13621054_1

2017228700 15 Sep 2017

7/17

Fig. 6

13621054 1

700

2017228700 15 Sep 2017

8/17

Fig. 7

13621054 1

2017228700 15 Sep 2017

9/17

50%

50%

Fig. 8a

Fig. 8b

70%

30%

810

870

Fig. 8c

13621054_1

2017228700 15 Sep 2017

10/17

920

Fig. 9

13621054_1

2017228700 15 Sep 2017

11/17

13621054_1

2017228700 15 Sep 2017

12/17

Fig. 11

13621054 1

2017228700 15 Sep 2017

13/17

Fig. 12

1200

13621054_1

2017228700 15 Sep 2017

14/17

1310

Fig. 13

13621054 1

2017228700 15 Sep 2017

15/17

Fig. 14

1400

13621054_1

2017228700 15 Sep 2017

16/17 ( ^Finish ) Fig. 15

13621054_1

2017228700 15 Sep 2017

17/17

Fig. 16

1600

13621054_1