I found an interview with Brad Cox that seems to hint that square brackets were for syntax:
DD: The square brackets are a distinctive part of Objective-C syntax. How did the square brackets come about and was this your doing?
BC: Yeah, that was me. Quite literally it was a search for something that wasn't taken. Curly braces were taken. Ordinary parentheses were taken. It was just a search for braces that wouldn't collide with something C used.
Tom Love also credits himself with the square brackets in Masterminds of Programming, but he seem to be saying that it was a stylistic choice:
I often refer to myself as the guy responsible for the square brackets in Objective-C, because Brad and I had a long conversation about. Do we have a C syntax that is consistently C, or do we create a hybrid language where I describe it as “the square bracket is a gear shift into the object land”? Our view was that if you had a hybrid language, you could build a set of foundation classes so that at some point most of the work is actually done inside the square brackets. This allows a lot of details to be hidden from a typical application programmer.
The square brackets are an indication of a message sent in Objective-C. The original idea was that once you built up a set of libraries of classes, then you’re going to spend most of your time actually operating inside the square brackets, so you’re really doing object-oriented programming using an underlying framework of objects that were developed in the hybrid language, which was a combination of procedural and object-oriented language. Then as you began to build up libraries of functionality, there’s less and less requirement to drop into the procedural world and you could stay within the square brackets. It was a deliberate decision to design a language that essentially had two levels— once you had built up enough capability, you could operate at the higher level. I actually think that’s one of the reasons. Had we chosen a very C-like syntax, I’m not sure anybody would know the name of the language anymore and it wouldn’t likely still be in use anywhere.