Individual Learning – Shadertoy

Last week I took a dive into Shadertoy, a website that allows users to write shaders and share them publicly, all within the browser. Many of the effects are truly awe inspiring with incredibly detailed, procedurally generated, 3D scenes ranging from lifelike forests as in the example below by Inigo Quilez (iq), the creator of shadertoy, to abstract and fractal, scenes like the following by Kali.

Rainforest	Generators

I created a more simple 2D shader rendering circles by following guides on youtube, mainly this one by iq https://www.youtube.com/watch?v=0ifChJ0nJfM

From there i started to draw more circles, adding some offsets using sine and cosine functions and i eventually came up with this cool perspective effect which makes a grayscale mask which i then mix between white and the default shader’s colour effect. (Click to see the shader on Shadertoy.com)

I continued to experiment further, using some of the other inputs that shadertoy gives you and also by taking in feedback from the comments (which came in very quickly) to create these next shaders.

	Using the Date input and the same circle drawing from the first shader to create a functional clock
	Pixelation effect to make make it look like the resolution of the canvas is changing and get a pixel art effect. This shader also includes mouse input to adjust the resolution and the blur on the circle
	A smoothstepped Value noise function is used to blend smoothly between two textures. Similar to an effect i have used in unity to blend cameras

Finally I took this one step further and implemented the first effect in a Unity shader so that it can be used as both a material on an object but also as a post process image effect. This wasn’t too difficult, even though Shadertoy uses GLSL and Unity used HLSL, they’re both very similar and most of the code worked fine bar a few vec2s needing to be float2s.

I hope you have enjoyed this week’s post, i certainly had a lot of fun researching the effects and demos that people have created using shadertoy, especially the 3D ones. If you would like to have a go at writing some shadertoy shaders i would recommend looking at iq‘s website here and The Art of Coding on YouTube.

Group Project 3

Last week lots of focus was put into getting an IOS build up and running. It was very time consuming as the Mac not only needed to update but it was running painfully slow, so installing the various tools took a very long time. It didn’t help that i needed ro reinstall Unity a few times as i initially had the wrong version, after that i had to reinstall once again as i could not get the IOS build package to install.

Following the trouble with getting everything installed, i spent the next few hours banging my head against the wall trying to build the XCode project using my free Apple ID. I was following this guide by Unity however the errors and buttons that the guide said should appear were not there for me and upon searching for the errors i was getting, all that i could find was posts made many years ago informing me that i needed to purchase a developer lisence (which the guide said was not needed for a test build).

The trouble i’ve had up until now regarding an IOS release has caused a lot of stress and i am now considering, for the sake of the game, no longer releasing on IOS and bringing forward either the android or PC releases.

I also swapped my Group project work on wednesday with my ILP work from thursday as i had recently been talking about shaders with one of my lecturers and i felt that it would be better for me to work on that while it was fresh on my mind and to finish the sprint the next day.

This week i will not be running a sprint as per my usual week so that i can focus on my written work. I am aware that this is a sizeable chunk of valuable time to be missing from the project, especially as i am now working on the polishing phase but i know from past experience that due to my learning disabilities that i need to allocate more time to written assignments and as the deadline approaches i feel it is more important to get a head start elsewhere so that i do not sacrifice marks on my other modules. I will however be playtesting the latest build at this week’s Ipswich Game Developer Meetup; If you are coming along i would love to get your feedback.

Group Project 2

Development went well this week, I have adjusted my sprint to run from monday to wenesday as I no longer need to work on Visual Scripting on Tuesday.

The UX changes I have made are small however they really make the game look and feel miles better than before and I now have the correct rights to the fonts in the game.

I also wrote a questionnaire for playtesters to give feedback on, I have learned that as i am in a university that questionnaires must be vetted by a Research Ethics board so that is being dealt with today to assure that my form is OK.

Somewhat worryingly I have previously written feedback questionnaires during my BA here and I and my classmates had not been aware of this, i am unsure of whether this is a big problem but i do now know this is the case and how to get it to the board.

Next week I will be adding analytics back in, using Google Firebase. I will also be working on more UX improvements as well as continuing to work on the IOS build.

Individual Learning – Snow Effect

This week I did some more work with vertex displacement and started work on an effect I had wanted to make for a while, even before I started writing shaders: Snow resting on top of an object.

Everything went pretty smoothly, in the vertex function i find the slope angle, if it’s flat enough for snow to be there i set a mask value to white and if not it’s black; I also displace the vert vertically.

Then in the fragment function I use the mask to lerp between the object’s texture and a snow texture (I left it white as I didn’t have one to hand). The effect is quite appealing though it does stretch the UVs around the displacement and the transition from snow to object can be jagged where there are sharp changes in geometry.

Here’s a video of the shader in action using a rock from the asset store for demonstration.

Once I had finished i decided to look into using tessellation which would give me more vertices to work with and in theory a better result. I found this page on the documentation which demonstrates how to implement tessellation in a surface shader.

I had been writing individual vertex and fragment shaders so I would have to re-write them however it did show me that I could perform vertex functions inside a surface shader using the vertex:vertFunction define in my shader code.

I used the Surface shader examples page on the unity docs to help me with my syntax and quickly got this very similar result the only major differance was that i could now store my mask value as a float in the Input struct and not a float4 in the v2f struct.

I did manage to get Tessellation and Phong Tessellation working without the snow effect but whenever I tried to include my vertex function I would get an error which i assume is caused because unity uses the vertex modifier after tessellation as it would usually be used to perform some sort of height map displacement. The tessellation was surprisingly easy to get up and running though.

This slideshow requires JavaScript.

The next step I would like to take is finding a way to use the tessellation with my snow effect to hopefully see an improvement compared to the original shader.

Individual Learning – Vertex Displacement

Last week I wrote a vertex displacement shader that uses a heightmap to displace vertices on a mesh. I managed to get a simple terrain example working very quickly by using an existing heightmap downloaded from google Images.

Here’s what it looks like with a standard plane:

I also created a higher resolution plane in Max to get this more detailed look. It is probably possible to use Tessellation to get the same result at runtime.

Following this i decided to experiment with Unity’s Perlin noise function using what I had learned from creating my own procedurally generated terrain in Monogame. It’s not a very practical approach to terrain generation but it was interesting to experiment with. I also added some colour and a water plane.

$VertexDisplacement_fractalterrain$

I had some trouble with the noise generation as I discovered Unity’s perlin noise function returns the same value at integer values but after i added some values to tweak the scale I was using floats anyway. I also had an issue where the texture didn’t appear to generate as I was getting no displacement or colour on the plane, this however was because I had forgotten to call Texture.Apply() after setting the pixels.

It would be interesting to explore the use of tessellation to skip having to create a pre-subdivided mesh.

Heightmap available here.

Group Project – 1

This week I felt that I had lost direction with the project however after a meeting with my course leader where we discussed the current state of the game and critically analysed the game objectively, through the eyes of a new player, I now have a good idea of the direction I need to move the project in.

The reason for my loss of direction is likely due to the game surpassing my original vision and clouding what really needs to be happening to turn my game, a minimal viable product, into a great game. The discussion has allowed me see what needs to happen now that the polishing phase has begun.

In the next sprint i will begin the polishing phase of the game. My tasks are focussed on streamlining the user experience and removing aspects of the game that are left over from the original game jam submission that are no longer correct.

Visual Scripting – Final Post

Good progress has been made continually throughout the project, I have run into a few problems but I have prioritised more important features rather than grinding away at the same problem wasting precious time. I am happy that the tool in its current state can produce a natural looking result and shows a lot of potential for further work.

The project is available on my Github here.

To Open the project from Github on your computer you will need the following:

Unreal Engine 4, Version 4.18.3
Visual Studio 17
- Windows 10 SDK
- Windows 8.1 SDK
- .NET desktop development
- Desktop development with C++
- Game Development with C++

I would have liked to have created a Windows build to use as a demo however I got stuck trying to fix a compile error that was not present when using the Compile button in the editor but was present in both Visual Studio and when packaging for windows. In the end I decided that it would be a waste of time spending hours debugging it rather than making sure it’s finished.

Further work

It’s very slow to generate so I would like to optimise everything, specifically getting the mesh data from the terrain and generating the normal.

Currently all of my code is in the Level blueprint so I would like to put it into it’s own blueprint class so that it’s easier to use in multiple levels.

I would like to add support for more biomes with differing settings and even moisture simulation to procedurally determine the location of biomes as per my initial proposal.

I would like to explore the possibility of creating my own heightmaps using Fractal noise, building on what I learned from implementing it in Monogame in my spare time, see Visual Scripting – Spawning Trees.

Finally I would like to use Noise to determine where vegetation should spawn as opposed to the current random number generator; this could possibly allow for me to further speed up the generation time as well as allowing more control to the user for where vegetation should appear.

Asset Ownership

Tree Mesh, Created in 3DS Max

Materials, Created by playing around in the material editor
• Snow Material: Noise node based off world position
• Ground Material: Picks Stone or grass texture dependant on the slope

Sourced Textures, From http://www.Textures.com:
• Grass0057_1_S
• TexturesCom_Cliffs0356_1_seamless_S
WaterPlain0017_1_S

Visual Scripting – UI Implementation

This week I added Menu to the tool so that the user can configure various Generation settings with sliders and such, the settings are:

Seed
Maximum Slope
Snowy biome height
Water Height
Tree Density

It is a little slow to generate as there is lots of room for optimisation however for demonstration purposes it’s fine.

Here is some footage of it working in the editor.

Most of my time getting it working was spent trying to communicate between the HUD blueprint and the Level blueprint I settled on having a reference to the HUD on the Camera object and searching for the camera in the level blueprint. From there I used an event dispatcher to allow the Menu Widget on the HUD to call the generate event. Figuring out how to set up the event dispatcher took a while as most of the forum posts explaining them would contain two screenshots of the node setup but not which blueprint should contain what.

Visual Scripting – Notable Problems

This post covers a few of the most notable problems I have had with UE4 regarding random crashes and compiling issues. I am less so concerned about the crashes themselves but more of the lack of supporting documentation surrounding fixes for what to me seems like a fairly common use of the engine.

There was a point where I had spent the day trying to debug a function that did not appear to be getting called. I went through my usual process of placing Breakpoints only to find that none of my variables had been declared, upon further investigation it became clear that the Compiler had taken issue with some pointer arrays, decided the function changed nothing and optimised it all away. There were no errors showing that there was an issue and it took an entire day to figure out the source of the problem and to fix it.

I wanted to move all of my generation code from the level blueprint and into its own class, either a blueprint or a C++ class but I just ended up spending hours trying to fix obscure linker and compile errors informing me that functions I had previously using did not exist. Also, whenever I compiled the code using the button in the editor it would crash without warning. The Documentation says nothing about fixing what I would assume are common errors nor how to properly set up the build file or even which header files should be included when working with certain classes. Furthermore it’s incredibly difficult to debug your code as it is not easy to tell if the error is coming from C++ or from Unreal Engine. It is also often the case that errors will be reported after one compile that will be gone after the next compile without a trace which is possibly more infuriating.

Finally I’d like to mention for anyone else who wishes to work with landscapes in the future. The editor will crash if you are creating a blueprint and try to set a child actor’s class to a Landscape; the editor will freeze for a second and then crash and present the error reporting window, the same as the compile bug mentioned earlier. If you have a lightning fast computer this might not be that much of a problem but if like me it takes you the best part of 5 minutes to launch UE4 this can really eat up your time and it certainly drained my motivation to continue.

Individual Learning – Toon Shader

This week I continued some work I started over Christmas that consisted mainly of reading through Unity’s Shader documentation and downloading a few assets from Turbosquid to play with as I start writing shaders.

I have decided on using Unity’s “Surface Shader” which combines Vertex and Pixel shaders into one, and does some clever compilation behind the scenes to create the individual vertex and pixel programs. They also allow you to use either built in lighting models or to write your own lighting functions, which I have used for the following shaders.

After a little bit of messing around set out with the goal of creating a toon/cel-shader. Toon shaders come in many forms but commonly they feature more flat shading and hard borders between light areas and shadow. I wrote two shaders: a colour ramp shader which samples a ramp texture, usually a gradient, and no diffuse texture to determine what a fragment colour should be when lit; the second shader I wrote uses hard coded light levels to create the effect while still sampling a diffuse texture.

Colour Ramp

Here you can see a a mesh on the left using a more standard Diffuse/Specular lighting model (lambert), and on the right, my modified mesh using a series of different ramps to colour the individual mesh parts. The ramp texture works by using the light’s intensity at each pixel to sample a texture, the brighter the intensity, the further left is sampled and the darker tones are found on the right.

The colour Ramp approach requires separating a mesh into multiple parts so that different materials can be applied for each ramp. You can see the individual ramps I created below and how I modified the mesh in 3DS Max

One the one hand I like how minimal and simplistic the mesh looks using just the colour ramp however as the mesh had one, I added a normal map to the shader which massively improved the look of the material. I also added it to the standard material on the left for comparison along with some colourful point lights to test how the lights tinted the surface.

This slideshow requires JavaScript.

Textured Toon

Next, I replaced the Lambert light function on the standard shader with my own custom function, which hard codes 3 light levels (full bright above 50%, half bright above 25% and a quarter below that) that are multiplied by the diffuse texture to create a similar but textured effect on the left.

While i was testing the the shader on other meshes i realized that the normal maps could be too strong and cause the surface to look ugly and noisy. To fix this I added a slider to the shader that lets me set the strength of the normal map the same way as Unity’s built in standard shader.

As you can see this gives me artistic control over how the normal map is affecting the surface.

Bonus Fun

For a bit of fun and as a bit of testing i dragged over the Sponza mesh from my Rendering Engine, threw some garish point lights into the scene and slapped my Textured Toon shader onto the walls, which, to me, looks pretty cool.

My Thoughts

I think that both shaders look interesting and give a unique feel each however they both have some technical annoyances that may get in the way. Firstly, the textured shader, while it does not require multiple Ramp textures and can use a mesh without being edited does currently have everything related to the lighting hard coded so if I want to tweak any of the settings I have to change magic numbers in the shader code. I could probably fix this by creating a grey-scale Light ramp similar to the ramped shader.

Secondly the Colour Ramped shader requires a number of individual textures to be created for a mesh, dependent on the number of colours that it requires which is a pain for this simple mesh but would be impractical for a more detailed object. However the artist can do any number of things with a ramp, including shifting the hue of the colour as it moves towards the darker shades or even use completely different colours which could lead to a number of interesting effects.

In the future I would like to delve into Unity’s GUI Editor scripts which allow you to set how the properties are laid out in a material and I imagine even add a gradient field to it which would remove the need to create the ramp textures externally.