Troubleshooting

From Bakery GPU Lightmapper: Wiki
Revision as of 11:23, 31 October 2021 by Mr F (talk | contribs) (I get a “Can't create texture” or "Can't create rasterizer state" error)
Jump to navigation Jump to search

This is a list of common problems, their symptoms and solutions.

Contents

Shadowmasks are not baked

Check following things:

  • Bakery's Render Mode is set to Shadowmask.
  • Masked lights have both Unity and Bakery light components on them.
  • Masked Bakery light components have "Baked contribution" set to "Indirect and shadowmask".

Shadowmasks are baked, but Unity renders real-time shadows instead

You're in Distance Shadowmask mode, it's expected. Read more about the difference:

https://docs.unity3d.com/Manual/LightMode-Mixed-Shadowmask.html

https://docs.unity3d.com/Manual/LightMode-Mixed-DistanceShadowmask.html

  • To switch between two modes, you can use "Distance Shadowmask" checkbox in Bakery main window.
  • If you're on 5.6, go to Window->Lighting->Mixed Lighting->Lighting Mode to switch the mode.

I can't see specular lighting after the bake

Apparently specular lighting was provided by real-time lights before the bake. Now the lighting is baked into static textures and there is no specular. However, you can:

  • Use Reflection Probes. Good option for highly glossy objects.
  • Use mixed lights and only bake GI/Shadowmasks. Specular remains real-time.
  • Bake in Dominant Direction or SH mode and use Bakery shaders with "Lightmapped specular" option. It will give some specular approximation. Looks good on surfaces with medium/high roughness.

Using a version control system I copied the scene to another PC and it doesn't look right

Make sure you followed version control guidelines.


Baking same asset in one scene breaks its UVs in another scene

The reason here is UV padding adjustment. It modifies the UVs in the asset, so spacing between UV charts is optimal, meaning they never leak over each other and there is no much wasted space. Unity has a manual Pack Margin option on model assets, but it’s limited since it applies the same exact value for all meshes inside the model, while each mesh can get drastically different size in the final lightmap based on its surface area. Therefore Bakery overrides it with automatically calculated per-mesh (instead of per-model) padding. However, because “optimal” varies depending on baking resolution, it can be problematic to reuse the same asset in isolated scenes with different lightmap size. There are multiple ways to solve it:

  • Generate your own optimal UVs. UV padding adjustment only applies to models with auto-generated UV layout (aka “Generate Lightmap UVs” checkbox) and never touches custom data.
  • Use UV padding: increase only checkbox and first bake the lowest resolution scene, and then the rest.
  • Just disable UV padding adjustment. Results will be similar to built-in baking.
  • Use two different models for different lightmap resolutions, so each will get the most efficient set of UVs.
  • Bake both scenes together.
  • Lock UVs using Save UV padding to asset

Denoiser throws error 505 or Unknown error

Update the driver.

If you are using a Kepler GPU, try switching to legacy denoiser.


I get an “Out of memory” error

This means rendering needed more video memory from your GPU than it has available.

First, try closing any other graphics-intensive programs (browsers too).

If the problem persists, the most likely reason is that the resolution you selected is unreasonably high for the given scene. Resolution is defined by Texels per Unit (this section also has a list of reference values) or set explicitly in Lightmap Groups. Reducing it will help. You can always selectively use high resolution for important objects via Scale in Lightmap.

If you have terrains in your scene, make sure Terrain optimization is enabled.

If you have a very high amount of painted trees on your terrain, try disabling the Export terrain trees option.


I get a “Can't map texture” error

Typically happens if your system went out of RAM (system memory, not video memory) during the scene export process. Your scene could already be taking too much RAM by itself, not leaving enough free memory for Bakery export.

  • Try closing any memory-intensive programs before baking.
  • Try reducing Max Resolution from 4096 to 2048.
  • If you are baking very highly detailed (millions of triangles) geometry, try excluding some of it from the bake.
  • Try disabling Export terrain trees.
  • Try enabling Terrain optimization.

This error can also be shown in case of GPU driver crash, in which case Unity editor also won't render anything.


I get a “Can't create texture” or "Can't create rasterizer state" error

Typically errors like this are caused by such chain of events:

  • Some objects in the scene have incorrect lightmapping UVs. Often the UVs contain many large triangles overlapping over each other.
  • Scene uses large lightmaps.
  • Bakery tries to rasterize a huge amount of 4k-sized overlapping triangles to prepare the UV GBuffer.
  • This operation takes > 2 seconds, and driver is reset by windows due to TDR: https://docs.microsoft.com/en-us/windows-hardware/drivers/display/timeout-detection-and-recovery
  • Consequent GPU operations are failed due to reset.

Solution:

  • Make sure all your objects either have non-overlapping UV2 or enable "Generate lightmapping UVs" on all model assets.

After importing Bakery I get errors containing “obsolete” and “UnityUpgradable”

Before importing Bakery, make sure you don't have any compilation errors in your project.

Restart Unity, reimport the package. Click “I made a backup, go ahead” if Unity asks.

See this page for more details.

It is also known that some versions of Unity Hub have a bug preventing API updater from running.


I get a "Can't load" error

If Bakery says it can't load something, make sure you have enough free disk space on the drive used for temporary data.

Some parts of Bakery can't deal with non-ASCII characters (this is going to be fixed soon). Try naming your scenes and lightmap group assets with ASCII characters. Latin letters and numbers must always work.


I get a "Launch timeout" error

This error can happen if your GPU took more time than it is allowed by Windows TDR.

  • Try lowering the value of Tile Size, especially if it's higher than 512, so baking job is split into smaller chunks.
  • Depending on the pass being processed, try lowering the Samples value (e.g. GI samples, Skylight samples, etc).
  • Reduce geometric complexity of the scene, if tile size/samples don't help.
  • Try disabling Terrain Optimization.
  • As a last resort, disable TDR registry keys. This will prevent Windows from resetting the driver, but your computer may appear unresponsive during the bake.

Additional instructions for changing the timeout (from Substance)


I have an Android project, and lightmaps don't look good

Android banding artifacts
Lightmap encoding in player settings
Build texture compression settings

Sometimes, when working on Android projects, you may encounter banding artifacts, with gradients having discrete steps of different hue. This is caused by texture compression and HDR encoding Unity uses for this platform, which often turns out to be "Double LDR" (see details). The problem is not exclusive to Bakery and will as well happen with built-in lightmaps.

In Player Settings, set Lightmap Encoding to High Quality.

If your target devices support ASTC compression, make sure it is enabled in Build settings. PVRTC is also a good choice, although it is mostly supported on iOS devices. ETC compression gives poor results for lightmaps.

If above solutions are not enough, select lightmap assets and set texture compression quality to High or None (try both).

Note that setting it to None will use more video memory on the device.


I get too many lightmaps

Possible causes are:

  • Atlas packer is set to Default or Post-packing is disabled. Set Atlas Packer to xatlas and enable Post-packing for best results.
  • Texels Per Unit value is high or Max Resolution is low. Bakery packs objects from open scenes into multiple lightmap atlases and packs them as rectangles (in a fashion similar to this; unless Hole Filling is enabled) with their size dependent on (surface area * texels per unit). If some object can't fit into existing atlases, a new one is allocated. New atlases are always as large as the Max Resolution parameter, or smaller (but not smaller than Min Resolution), if the estimated surface area of remaining unpacked objects is below some threshold. You can also increase Min Resolution a bit, replacing a bunch of small (but dense) maps with a few large (but possibly with some empty space).

If not using post-packing:

  • If you bake multiple open scenes and Split by scene checkbox is on, different scenes will never share the same lightmap, therefore multiplying their count.
  • If Terrain optimization is on, terrains will always use a separate lightmap.

Pre-atlasing your scene in your modelling package of choice and using the Original UV mode instead of relying on the auto-atlaser can significantly improve packing effectiveness.

Since v1.71 Bakery prints number of future used lightmaps to console when enabling/refreshing Checker Preview. You can experiment with resolution/density settings until the desired value is printed.


I get jagged shadows

Jagged shadow mitigation

Apparently you are baking very sharp shadows at low resolution. Try increasing shadow spread parameter on the light (direct or point) and optionally enable bicubic interpolation.


"Exporting scene - preparing" is taking too long


Stuck at "Waiting for Unity to initialize the probes..."

There isn't much Bakery can do about the logic of Unity lighting data asset generation (there is no API in the engine to control it at the moment). Try to focus editor window (especially scene view) to wake Unity up.


Baked prefabs instantiated at runtime in a build have no lightmaps

Because Unity technically has no support for prefabs having lightmaps, it can omit lightmapped shader versions from builds, not realizing they are needed. You can fix it by going to Shader stripping settings and enabling lightmap modes you use. Additionally, try disabling Optimize Mesh Data in Player Settings, as Unity may omit UV from the build.


Speedtrees look fine in the editor, but black in builds

Also related to shader stripping. See the prefab solution.


HDRI produces scattered bright spots (fireflies)

Typical "fireflies"
Removing sun from HDRI

This artefact comes from the fact your HDRI has very bright, yet very small parts, typically sun. Rays will miss it often, producing occasional spots when they don't. This problem is well known in many offline ray tracers as well.

Similar problem may also occur when using very small and very bright emissive surfaces, or similarly small surfaces lit by a very intense light. You can use Light Meshes instead of small emissives to prevent it.


"Adjusting UV padding" happens too often and takes too much time

UV padding adjustment is a feature that optimizes asset UVs based on the size they take in lightmaps. This process will happen for a model if:

  • Option is globally enabled.
  • Asset has "Generate Lightmap UVs" checkbox enabled. Models with custom UVs are not adjusted.
  • Model was not adjusted before or...
  • ... its lightmap size has changed.

Therefore it can be triggered for all auto-unwrapped models after their geometry or resolution changes. UV padding adjustment will try to waste less lightmap resolution and reduce texel leaking, but it is not necessary. If you are doing many experimental/draft bakes, you may consider disabling it (set to "Don't change"), then possibly enabling it back for the final quality bake.

Also note that using xatlas as Unwrapper can be notoriously slower comparing to Default.


Denoiser produces bright edges around geometry/shadows

Bright edges

Sometimes you may encounter bright edges or spots at high contrast edges caused by the denoising neural net, most notably OptiX 6 and 7.

  • Use OptiX 5 or OpenImageDenoise if you can (denoiser option).
  • Try enabling Denoise: Fix bright edges.
  • Try setting texture compression to None on lightmap assets.
  • Try setting Backface GI to 1, removing dark shadows inside objects. While these shadows are normally not visible, their edges may cause problems for both the denoiser and texture compression.



Light probes are too bright in Subtractive mode

The effect of occlusion probes

Light occlusion information is not baked into probes. Solution:

Some versions of Unity may need a scene reload to properly apply the Lighting Data Asset.


Baking produces "Not allowed to access uv2" errors

All lightmapped models must have Read/Write enabled on.


Whole scene is very bright or black

  • Check if your light intensities are in a reasonable range (< 100).
  • Check the same for indirect intensity on all lights. A multiplication of direct and intensity intensity must not go into thousands, otherwise GI algorithm may overflow.
  • Check if you don't use assets with High mesh compression. When set to High, Unity mesh compression will completely destroy lightmap UVs, only leaving 8 bits for them. This may produce many lightmapping errors including this one.

Bakery says "Device is not DX11"

Currently Bakery needs Unity editor to be running in DX11 mode to directly exchange texture data, so make sure your editor's window header says "<DX11>". If it does not, change active graphics API to DX11:

  • Go to Player Settings.
  • If "Auto Graphics API for Windows" is unchecked, you will see a list of APIs.
  • Make sure "Direct3D11" is on top or place it there.

Bakery doesn't work on my 3070/3080/3090 GPU?

1. Please enable RTX mode.

2. If denoiser also fails, please update from github and use OpenImageDenoise option (some instructions are here).

Updated denoiser (both new OptiX version and OpenImageDenoise) as well as automatic hardware detection will be soon released on the Asset Store.


Denoiser produces grid-like patterns

Grid-like denoiser pattern

OptiX denoiser might produdce grid-like patterns when fed with very low-precision lightmaps. Low precision is usually caused by either extremely dark or extremely bright lighting. It is recommended to stay within reasonable intensity range (see a similar advice for HDRP above).

  • Alternatively use OpenImageDenoise, as it does not produce this pattern.
  • This issue also does not affect OptiX 5.1 denoiser ("Legacy denoiser") as its training dataset is in its own DLL, but affects the non-legacy version (OptiX 6.0) and only on driver versions newer than 442.50, as the training datasets are in the driver (explanation from NVIDIA).

I get "rtcRes == RTC_SUCCESS" error

You are using a 3xxx GPU with RTX mode disabled. Enable RTX mode.


Baking any scene crashes with unknown error

There were some reports that Avast antivirus can block Bakery from running, even though Bakery's executables don't trigger any antivirus (including Avast) when scanned via VirusTotal. Perhaps it just block any executables that were launched 'indirectly', i.e. from another application. In any case, if you have Avast, try adding your project folder to exception list.


Combined meshes (e.g. using Mesh Combine asset) bake incorrectly

Mesh Combine "Regenerate Lightmap UVs" option

Most mesh combining assets have an option to regenerate lightmap UVs. Without it, combined UVs will overlap each other. If you are using Mesh Combine, make sure "Regenerate Lightmap UVs" option is enabled in output settings.


I get a "Can't create vertex buffer" error

This error can happen if the scene has more than ~40 million vertices. It is recommended to reduce geometric complexity of the scene, as tracing it would be too slow anyway. The typical cause of going over this limit is trying to bake with large arrays of trees painted on the terrain. In this case try disabling Export Terrain Trees.


Soft shadows appear like multiple hard shadows

Undersampled shadow

Increase Shadow Samples on the light or decrease the Shadow Spread.


Lightmaps are black on GTX 650

Apparently newer Nvidia drivers deprecate some CUDA functions on older hardware. It is recommended to use driver version 442 on GTX 650, as it still supported everything Bakery uses.