29

TL;DR

Before

SWIFT_WHOLE_MODULE_OPTIMIZATION = NO

  • Debug compile takes 10-15 minutes
  • Release compile takes 25+ minutes
  • po works fine in LLDB

After

SWIFT_WHOLE_MODULE_OPTIMIZATION = YES

  • Debug compile takes 1-2 minutes
  • Release compile takes ~8 minutes
  • po always causes Xcode to crash

Any idea why the horrible compile times based on this info, and/or why Xcode might be crashing?

Deets

I'm working on a large 100% Swift project (there are 3rd party libraries in Objective-C, but all our code is Swift). We have been having atrocious compile times, usually around 10-15 minutes to compile the debug configuration and 30+ minutes to compile the release configuration.

This project has been very difficult to work with because of the horrible compile times. I've been searching for ways to improve this, particularly through build settings and for months had no luck. One thing I overlooked was SWIFT_WHOLE_MODULE_OPTIMIZATION, particularly because any mention of it claims it will increase a project's compile time.

So the other day we enabled SWIFT_WHOLE_MODULE_OPTIMIZATION and lo and behold we have a 10x improvement on compile times.

The problem is, now whenever we're debugging the project and try printing an object in lldb with po myObject Xcode immediately crashes. Here's some info from the crash log:

Process: Xcode [5860]
Path: /Applications/Xcode.app/Contents/MacOS/Xcode
Identifier: com.apple.dt.Xcode
Version: 6.4 (7720)
Build Info: IDEFrameworks-7720000000000000~8
App Item ID: 497799835
App External ID: 812725084
Code Type: X86-64 (Native)
Parent Process: ??? [1]
Responsible: Xcode [5860]

Date/Time: 2015-08-05 15:53:08.265 -0600
OS Version: Mac OS X 10.11 (15A235d)
Report Version: 11

Time Awake Since Boot: 13000 seconds

Crashed Thread: 20

Exception Type: EXC_BAD_ACCESS (SIGSEGV)
Exception Codes: KERN_INVALID_ADDRESS at 0x000000000000008f
Exception Note: EXC_CORPSE_NOTIFY

VM Regions Near 0x8f: --> __TEXT 000000010ef62000-000000010ef63000 [ 4K] r-x/rwx SM=COW /Applications/Xcode.app/Contents/MacOS/Xcode

Application Specific Information:
ProductBuildVersion: 6E35b

Here's the stack trace on the crashed thread:

Thread 20 Crashed:: <DBGLLDBSessionThread (pid=6402)>
0   com.apple.LLDB.framework        0x0000000116b09ab4 swift::ArchetypeBuilder::resolveArchetype(swift::Type) + 68
1   com.apple.LLDB.framework        0x0000000116b0f808 std::__1::__function::__func<substConcreteTypesForDependentTypes(swift::ArchetypeBuilder&, swift::Type)::$_6, std::__1::allocator<substConcreteTypesForDependentTypes(swift::ArchetypeBuilder&, swift::Type)::$_6>, swift::Type (swift::Type)>::operator()(swift::Type&&) + 152
2   com.apple.LLDB.framework        0x0000000116bc0986 swift::Type::transform(std::__1::function<swift::Type (swift::Type)> const&) const + 54
3   com.apple.LLDB.framework        0x0000000116bc0f2b swift::Type::transform(std::__1::function<swift::Type (swift::Type)> const&) const + 1499
4   com.apple.LLDB.framework        0x0000000116bc0bbb swift::Type::transform(std::__1::function<swift::Type (swift::Type)> const&) const + 619
5   com.apple.LLDB.framework        0x0000000116bc0c0a swift::Type::transform(std::__1::function<swift::Type (swift::Type)> const&) const + 698
6   com.apple.LLDB.framework        0x0000000116b0c8f2 swift::ArchetypeBuilder::substDependentType(swift::Type) + 50
7   com.apple.LLDB.framework        0x0000000116e9554e (anonymous namespace)::LowerType::visitAnyStructType(swift::CanType, swift::StructDecl*) + 270
8   com.apple.LLDB.framework        0x0000000116e92e66 swift::Lowering::TypeConverter::getTypeLoweringForUncachedLoweredType(swift::Lowering::TypeConverter::TypeKey) + 150
9   com.apple.LLDB.framework        0x0000000116e92b39 swift::Lowering::TypeConverter::getTypeLowering(swift::Lowering::AbstractionPattern, swift::Type, unsigned int) + 2361
10  com.apple.LLDB.framework        0x0000000116f8f711 lldb_private::SwiftSILManipulator::emitLValueForVariable(swift::VarDecl*, lldb_private::SwiftExpressionParser::SILVariableInfo&) + 1521
11  com.apple.LLDB.framework        0x00000001172ac7ee (anonymous namespace)::LLDBNameLookup::emitLValueForVariable(swift::VarDecl*, swift::SILBuilder&) + 102
12  com.apple.LLDB.framework        0x0000000116ebb162 swift::Lowering::SILGenFunction::emitInitializationForVarDecl(swift::VarDecl*, swift::Type) + 98
13  com.apple.LLDB.framework        0x0000000116ebbc74 swift::ASTVisitor<(anonymous namespace)::InitializationForPattern, void, void, void, std::__1::unique_ptr<swift::Lowering::Initialization, std::__1::default_delete<swift::Lowering::Initialization> >, void, void>::visit(swift::Pattern*) + 404
14  com.apple.LLDB.framework        0x0000000116ebbc57 swift::ASTVisitor<(anonymous namespace)::InitializationForPattern, void, void, void, std::__1::unique_ptr<swift::Lowering::Initialization, std::__1::default_delete<swift::Lowering::Initialization> >, void, void>::visit(swift::Pattern*) + 375
15  com.apple.LLDB.framework        0x0000000116ebba0d swift::Lowering::SILGenFunction::visitPatternBindingDecl(swift::PatternBindingDecl*) + 45
16  com.apple.LLDB.framework        0x0000000116f0617c swift::Lowering::SILGenFunction::visitBraceStmt(swift::BraceStmt*) + 284
17  com.apple.LLDB.framework        0x0000000116ecd1c0 swift::Lowering::SILGenFunction::emitFunction(swift::FuncDecl*) + 320
18  com.apple.LLDB.framework        0x0000000116ea3966 swift::Lowering::SILGenModule::emitFunction(swift::FuncDecl*) + 246
19  com.apple.LLDB.framework        0x0000000116ea3828 swift::Lowering::SILGenModule::visitFuncDecl(swift::FuncDecl*) + 168
20  com.apple.LLDB.framework        0x0000000116ea579b swift::Lowering::SILGenModule::emitSourceFile(swift::SourceFile*, unsigned int) + 427
21  com.apple.LLDB.framework        0x0000000116ea5c22 swift::SILModule::constructSIL(swift::Module*, swift::SILOptions&, swift::SourceFile*, llvm::Optional<unsigned int>, bool, bool) + 386
22  com.apple.LLDB.framework        0x0000000116ea5d42 swift::performSILGeneration(swift::SourceFile&, swift::SILOptions&, llvm::Optional<unsigned int>, bool) + 98
23  com.apple.LLDB.framework        0x00000001172aa617 lldb_private::SwiftExpressionParser::Parse(lldb_private::Stream&, unsigned int, unsigned int, unsigned int) + 10715
24  com.apple.LLDB.framework        0x000000011706b3e8 lldb_private::ClangUserExpression::Parse(lldb_private::Stream&, lldb_private::ExecutionContext&, lldb_private::ExecutionPolicy, bool, unsigned int) + 1064
25  com.apple.LLDB.framework        0x000000011706cdb4 lldb_private::ClangUserExpression::Evaluate(lldb_private::ExecutionContext&, lldb_private::EvaluateExpressionOptions const&, char const*, char const*, lldb_private::SharingPtr<lldb_private::ValueObject>&, lldb_private::Error&, unsigned int, std::__1::shared_ptr<lldb_private::Module>*) + 628
26  com.apple.LLDB.framework        0x00000001171d1696 lldb_private::Target::EvaluateExpression(char const*, lldb_private::StackFrame*, lldb_private::SharingPtr<lldb_private::ValueObject>&, lldb_private::EvaluateExpressionOptions const&) + 376
27  com.apple.LLDB.framework        0x000000011716d75c lldb_private::SwiftLanguageRuntime::GetObjectDescription(lldb_private::Stream&, lldb_private::ValueObject&) + 668
28  com.apple.LLDB.framework        0x00000001170464e6 lldb_private::ValueObject::GetObjectDescription() + 370
29  com.apple.LLDB.framework        0x000000011548e228 lldb::SBValue::GetObjectDescription() + 76
30  com.apple.dt.dbg.DebuggerLLDB   0x00000001153f3c9e -[DBGLLDBDataValue _lldbValueObjectDescription] + 24
31  com.apple.dt.dbg.DebuggerLLDB   0x00000001153f3b7f -[DBGLLDBDataValue lldbDescription] + 29
32  com.apple.dt.dbg.DebuggerLLDB   0x00000001154023dc __87-[DBGLLDBSession printDescriptionOfDataValueToConsole:runAllThreads:completionHandler:]_block_invoke + 182
33  com.apple.dt.dbg.DebuggerLLDB   0x0000000115402e6d -[DBGLLDBSession handleNextActionWithState:withRunPending:] + 424
34  com.apple.dt.dbg.DebuggerLLDB   0x00000001153fdf44 DBGLLDBSessionThread(void*) + 980
35  libsystem_pthread.dylib         0x00007fff8ec12cb3 _pthread_body + 131
36  libsystem_pthread.dylib         0x00007fff8ec12c30 _pthread_start + 168
37  libsystem_pthread.dylib         0x00007fff8ec10419 thread_start + 13

Seems like we may be on the forefront of new technology here because I haven't found much help on this issue yet. I'm wondering if we have some sort of unusual configuration that is causing the compile time issues, or if there is a known reason why lldb might be crashing. Its the same on multiple different machines, with El Capitan, Yosemite, Xcode 6.3, Xcode 6.4. Any help is appreciated!

LunaCodeGirl
  • 5,432
  • 6
  • 30
  • 36
  • 1
    It's weird that enabling Whole Module Optimization decreases compile times. Apple says that compilation with enabled Whole Module Optimization should take longer. Because with this it can't compile multiple files at once, because it should look at all files in the module. – Juri Noga Aug 11 '15 at 07:01
  • Yea that's what's really odd to me. We have ~500 files in the project and they're each taking a few seconds to compile with Whole Module Optimization turned off. – LunaCodeGirl Aug 11 '15 at 18:35
  • 1
    It's actually recompiling every file every time you build? It shouldn't be doing that if it is. – rcw3 Aug 14 '15 at 20:27
  • @rcw3 it does with SWIFT_WHOLE_MODULE_OPTIMIZATION = YES, and it frequently does with it turned off, but it definitely depends on what files you're working in. Certain model files seem to cause the whole project to recompile every time they're changed. Other files are fairly isolated and partial recompile works as long as you don't change too much. – LunaCodeGirl Aug 14 '15 at 20:35
  • 1
    That seems to be expected with SWMO=Yes.... BTW, I'm guessing this is a wild goose chase, but how much memory do these machines have? – rcw3 Aug 14 '15 at 20:38
  • 16GB! And 3ghz i7 processors. See for yourself: https://www.evernote.com/l/AAXzDN0bN5lEJ5W5yNcl_1910NRVokUoFssB/image.png – LunaCodeGirl Aug 14 '15 at 21:05

2 Answers2

11

I've worked and I am actively working on projects that are probably even larger than yours (50+ libraries, hundreds of custom classes with thousands of LoC). It takes few seconds to compile and up to 2 minutes for release build (Late 2013 MBP), even for full Swift project.

I would highly recommend to look for other causes, as what this option tells you is that your problem is somewhere else since it affects app performance, not building speed. Maybe you have your .pch file full of stuff that does not belong there or something, it can cause this kind of slowing.

Other interesting thing is use of inference. I personally declare type for every single variable just because it is easier to read for others, but there is more to it. Since the inference can get complicated, it can cause compiler to take A LOT of time to figure out what your code actually does. Read this example article, though it was more like alpha-problem. BUT MAYBE, just maybe, you have something like this in your code as well. What would be worth doing is trying to remove all the swift code, compile libraries if that works well, then you know the problem is in your swift code and it will be probably connected to this.

Also, if you provide WHAT actually slows your build time (because you can see whole build process with timestamps, and there is high probability that one step will just take forever), it can help pinpoint the problem more accurately.

Hope it helps!

Edit: Another article on this interested topic

Jiri Trecak
  • 5,092
  • 26
  • 37
  • > because you can see whole build process with timestamps Do you mean in Xcode? If so then its the compilation step that takes forever. When WHOLE_MODULE_OPTIMIZATION turned off I can see each swift file compiling and it take about 2-4 seconds per file. I also build using xctool, which gives me the compile time per file in milliseconds, and they were almost all about 2000 milliseconds except for a few outliers, but the longest was 6000 milliseconds, so nothing dramatic. Is there another way I can see more granular steps and times? – LunaCodeGirl Aug 13 '15 at 20:18
  • 1
    Please try this post as I am now on mobile and more elaborate answer can be found there. Try finding the most expensive file using xctool -jobs 1, and see if there is some extreme somewhere – Jiri Trecak Aug 13 '15 at 20:30
  • There aren't any outliers in the compile time....a few take 6 seconds or so but almost every single file takes 2 seconds or longer. – LunaCodeGirl Aug 14 '15 at 16:31
  • 1
    Best answer so far but was hoping for some more definite reasons why we might be seeing this issue. – LunaCodeGirl Aug 17 '15 at 16:45
  • 1
    While the advice to write all types is good to minimize compilation time, it's actually a disaster for code readability. It makes sense to declare type when declaring class attributes (e.g. `var x: Bool = false` instead of `var x = false`) but definitely it does not make sense to declare a type for every closure or local variable. Also note that the major problem with compilation time is that the compiler cannot see dependencies between files and compiles every file multiple times. That can be currently solved by the whole module optimization trick. – Sulthan Jan 27 '18 at 19:01
9

I managed to fix this in our project. (12 mins total compile time)

The key is to set the SWIFT_WHOLE_MODULE_OPTIMIZATION = YES as you say in the user defined settings

But you need to do one more change. You need to set the optimization level to None in debug otherwise you will not be able to debug your app.

I wrote a blog post about it here:

https://tech.zalando.com/blog/improving-swift-compilation-times-from-12-to-2-minutes/

João Nunes
  • 3,751
  • 31
  • 32