Metal Programming Guide

Description

Janie Clayton is an independent iOS developer and graphics programmer. Janie is the author on several books on iOS and Swift development, including the Swift Apprentice and iOS 10 SDK. She records her journey down the rabbit hole on her personal blog at  http://redqueencoder.com. Janie lives outside of Madison, Wisconsin, with her attempted grumble of pugs and multitude of programming books.

• Fills a major gap, teaching Apple’s exciting new technology for high-performance 3D graphics, gaming, scientific programming, visualization, big data, and machine learning
• Packed with hands-on projects and real code samples covering everything from simple draw calls to advanced parallel computing
• Includes uniquely thorough coverage of data-parallel programming and performance/debugging tools, not just 3D graphics
• Author hand-picked by Apple’s Metal team, with tech review by world-class graphics programming expert Dave Shreiner

Metal enables Apple iOS and macOS platform developers to maximize performance in demanding tasks like 3D graphics, games, scientific programming, visualization, and now GPU-accelerated machine learning. Apple’s Swift + Metal combination runs faster than nearly any other language used in these areas. Now, there’s an authoritative and 100% practical guide to making the most of Metal. Pioneering Apple developers Basel Farag and Janie Clayton cover everything from simple draw calls to advanced parallel computing, teaching through hands-on projects and industry-realistic code samples. They cover a wide array of applications, from 2D and 3D graphics to neural networking. Farag and Clayton offer a uniquely valuable perspective on high-performance data-parallel programming with Metal — a topic that has been woefully underserved by earlier resources. They also offer expert guidance on Metal performance and debugging, helping you capture every iota of performance the Metal API and shader language can offer.

History of Graphics APIs 4

Metal: The New Way to Do Graphics on Apple Platforms 6

Metal in Context: How Metal Complements and Supports Other Platform Frameworks 10

Summary 10

Chapter 2: Overview of Rendering and Raster Graphics 13

Representing the GPU 14

Preparing Data for the GPU 16

Summary 24

Chapter 3: Your First Metal Application (Hello, Triangle!) 25

Creating a Metal Application in Xcode (without Using a Template) 25

Creating a MTLDevice 28

Creating a CAMetalLayer 28

Creating a Vertex Buffer 29

A First Look at Shaders 31

Libraries, Functions, and Pipeline States 34

Introducing Render Passes 35

Introducing MetalKit Features and MTKView 39

Summary 40

Language of Mathematics 43

Coordinate Spaces and Moving among Them 44

Points, Vectors, and Vector Operations 46

Normalization and Unit Vectors 49

Pythagorean Theorem 50

Sine, Cosine, and Tangent 52

Matrices and Matrix Operations 53

Transformations: Scale, Translation, Rotation, Projection 55

Summary 60

Chapter 5: Introduction to Shaders 61

Metal Shading Language Overview 61

Setting Up Shaders 63

Your First Shader: Pass Through 63

Writing Your First Shader 68

Uniform Buffer 74

Summary 82

Chapter 6: Metal Resources and Memory Management 83

Introduction to Resources in Metal 83

The Argument Table: Mapping between Shader Parameters and Resources 84

Buffers 85

Resource Options: Storage Mode, Cache Mode, Purgeability 86

Preparing Data for the Vertex Shader and Vertex Descriptors 87

Copying to and from Buffers 88

Introduction to Textures 89

Copying to and from Textures 93

Compressed Texture Support 94

The Blit Command Encoder 94

Generating Mipmaps 96

Summary 97

Chapter 7: Libraries, Functions, and Pipeline States 99

What Are Libraries and Functions? 99

The Metal Two-Phase Compilation Architecture 100

Creating Libraries at Compile Time and Runtime 101

Command Encoders 103

Render Pipeline Descriptors and State 104

Pipeline Reflection 105

Summary 105

Chapter 8: 2D Drawing 107

Metal Graphics Rendering Pipeline 107

Sample Project: Build a Star 108

Metal Primitive Types 113

Responding to MTKViewDelegate Methods 115

Retrieving a Drawable 115

Creating a Command Buffer 116

Creating a Command Encoder 117

Fixed-Function State on the Command Encoder 119

Passing Data to Shaders 121

Issuing Draw Calls 124

Scheduling and Enqueuing Command Buffers 124

Summary 126

Chapter 9: Introduction to 3D Drawing 127

Model-View-Projection Transformations 127

Clip Space and the View Frustum 131

Shading Models 136

Basic Lighting 138

Animation 140

Summary 141

Chapter 10: Advanced 3D Drawing 143

Constructing a Hierarchical World with Scene Graphs 143

Instanced Rendering 145

Summary 155

Chapter 11: Interfacing with Model I/O 157

What Are Model Files? 158

Importing a Model 161

Meshes and Submeshes 163

Render State Pipeline 164

Asset Initialization 165

Render State Setup and Drawing 166

Exporting Files 167

Summary 168

Chapter 12: Texturing and Sampling 169

Texture Mapping 169

Mipmapping 171

Sampling 172

Precompiled Sampler States 175

Passing Textures and Samplers to Graphics Functions 177

Summary 182

Chapter 13: Multipass Rendering Techniques 183

When (and Why) to Use Multipass Rendering 183

Metal Render Pass Descriptors 184

Creating and Managing Render Targets 185

Revisiting Load-Store Actions 188

Summary 190

Chapter 14: Geometry Unleashed: Tessellation in Metal 191

Removing Bottlenecks by Using Tessellation 191

Catmull-Clark Subdivision 192

Per-Patch Tessellation Factors 195

Metal Tessellation Fixed-Function Pipeline 197

Setting Up a Tessellation Kernel 201

Post-Tessellation Vertex Function 202

Draw Patches 204

Summary 208

Introduction to GPU Programming 212

Concurrency versus Parallelism 213

Using GPUs for General Computation 216

Kernel Functions 217

The Metal Compute Command Encoder 218

Issuing Grids of Work 220

Finding Your Way in the Grid inside the Kernel Function 223

Reading and Writing Resources in Kernel Functions 223

Summary 224

Chapter 16: Image Processing in Metal 225

Introduction to Image Processing 225

Creating a Metal Texture 227

Desaturation Kernels 230

Convolution and Dispatching a 2D Grid 232

Blur Effects 235

Selecting an Optimal Threadgroup Size 238

Summary 239

Chapter 17: Machine Vision 241

How a Computer Sees the World 241

Noise and Smoothing 242

Sobel Edge Detection 244

Thresholding 245

Histograms 246

Facial Recognition 246

Summary 253

Chapter 18: Metal Performance Shaders Framework 255

Overview of Metal Performance Shaders Framework 255

Image Processing with the MPS Framework 257

Matrix Operations with MPS 267

Summary 269

Chapter 19: Neural Network Concepts 271

Overview of Neural Networks 271

Neural Network Components 272

Neural Network Architecture 281

Summary 283

Chapter 20: Convolutional Neural Networks 285

History of Convolutional Neural Networks 285

MPSImage 289

Convolutional Neural Network Kernels 290

Convolution Data Source 296

Neural Network Graph 298

Summary 299

Index 301

The first comprehensive, example-rich introduction to Apple’s powerful Metal API and shading language

• Fills a major gap, teaching Apple’s exciting new technology for high-performance 3D graphics, gaming, scientific programming, visualization, big data, and machine learning
• Packed with hands-on projects and real code samples covering everything from simple draw calls to advanced parallel computing
• Includes uniquely thorough coverage of data-parallel programming and performance/debugging tools, not just 3D graphics
• Author hand-picked by Apple’s Metal team, with tech review by world-class graphics programming expert Dave Shreiner

Additional information