I found that hamstergene's answer isn't correct. In fact, NSTextView highlights its text container bounds line by line.
So, if you use paragraph's head indents then the paragraph leading empty space will be highlighted. And if you select EOL character then the trailing space of the paragraph will be highlighted as well.
My solution was to nullify head and tail indents of the paragraph style (I cache them in the private variable and put them back when my text storage is accessed for printing) and simply adjust frame of the text container line via overrided lineFragmentRectForProposedRect: atIndex: writingDirection: remainingRect method of my NSTextContainer subclass.
But then I found much proper way. Just override func fillBackgroundRectArray(_ rectArray: UnsafePointer<NSRect>, count rectCount: Int, forCharacterRange charRange: NSRange, color: NSColor) of the NSLayoutManager, calculate your rects and call super with those rects. And if you calculated selection rectangles properly, you'll get the exact selection behavior like in Apple Pages or MS Word.
Simple and easy!
UPDATE
Here's my code for calculating selection rects:
public override func fillBackgroundRectArray(_ rectArray: UnsafePointer<CGRect>, count rectCount: Int, forCharacterRange charRange: NSRange, color: OSColor) {
// if characters are selected, make sure that we draw selection of those characters only, not the whole text container bounds
guard let textView = textContainer(forCharacterIndex: charRange.location)?.textView,
NSIntersectionRange(textView.selectedRange(), charRange).length > 0,
let textStorage = self.textStorage as? ParagraphTextStorage else {
super.fillBackgroundRectArray(rectArray, count: rectCount, forCharacterRange: charRange, color: color)
return
}
let selectedGlyphRange = self.glyphRange(forCharacterRange: charRange, actualCharacterRange: nil)
var selectionRectArray: [CGRect] = []
enumerateLineFragments(forGlyphRange: selectedGlyphRange) { (rect, usedRect, textContainer, glyphRange, stop) in
let lineCharRange = self.characterRange(forGlyphRange: glyphRange, actualGlyphRange: nil)
let intersection = NSIntersectionRange(charRange, lineCharRange)
// if selected all characters of the line, then we already have teir layout rects
if intersection == lineCharRange {
let paragraphIndex = textStorage.paragraphIndex(at: intersection.location)
let paragraphRange = textStorage.paragraphRanges[paragraphIndex]
let hasNewLineChar = lineCharRange.max == paragraphRange.max && paragraphRange.max < textStorage.length ||
paragraphRange.max == lineCharRange.max && intersection.max == textStorage.length && paragraphIndex < textStorage.paragraphRanges.count - 1
let newLineCharSize = hasNewLineChar ? self.newLineCharSize : .zero
let lineRect = CGRect(x: usedRect.origin.x + textView.textContainerInset.width + textContainer.lineFragmentPadding,
y: usedRect.origin.y + textView.textContainerInset.height - (rect.height - usedRect.height),
width: usedRect.width + newLineCharSize.width - textContainer.lineFragmentPadding * 2,
height: rect.height)
selectionRectArray.append(lineRect)
} else {
// calculate rect for partially selected characters of the line
let partialRect = self.usedLineRect(forCharacterRange: intersection, in: textContainer)
selectionRectArray.append(partialRect)
}
}
super.fillBackgroundRectArray(selectionRectArray, count: selectionRectArray.count, forCharacterRange: charRange, color: color)
}
public func usedLineRect(forCharacterRange charRange: NSRange, in textContainer: NSTextContainer) -> CGRect {
guard let textView = textContainer.textView, let textStorage = textStorage as? ParagraphTextStorage else { return .zero }
let glyphRange = self.glyphRange(forCharacterRange: charRange, actualCharacterRange: nil)
let textContainer = self.textContainer(forGlyphAt: glyphRange.location, effectiveRange: nil) as! ModernTextContainer
let paragraphIndex = textStorage.paragraphIndex(at: charRange.location)
let paragraphRange = textStorage.paragraphRanges[paragraphIndex]
let hasNewLine = paragraphRange.max == charRange.max && charRange.max < textStorage.length ||
paragraphRange.max == charRange.max && charRange.max == textStorage.length && paragraphIndex < textStorage.paragraphRanges.count - 1
let newLineCharSize = hasNewLine ? self.newLineCharSize : .zero
// if new line is in range, boundingRect will return the whole width of the text container, fix that
let noNewLineGlyphRange = hasNewLine ? NSRange(location: glyphRange.location, length: glyphRange.length - 1) : glyphRange
let charRect = boundingRect(forGlyphRange: noNewLineGlyphRange, in: textContainer)
let lineRect = lineFragmentRect(forGlyphAt: noNewLineGlyphRange.location, effectiveRange: nil, withoutAdditionalLayout: true)
#if os(macOS)
// respect the flipped coordinate system with abs function
let rect = CGRect(x: charRect.origin.x + textView.textContainerInset.width,
y: abs(charRect.origin.y + textView.textContainerInset.height - (lineRect.height - charRect.height)),
width: charRect.width + newLineCharSize.width,
height: lineRect.height)
#else
let rect = CGRect(x: charRect.origin.x + textView.textContainerInset.left,
y: abs(charRect.origin.y + textView.textContainerInset.top - (lineRect.height - charRect.height)),
width: charRect.width + newLineCharSize.width,
height: lineRect.height)
#endif
return rect
}
The important part of this extremely fast-performed calculation is that I'm using my own ParagraphTextStorage implementation. Its purpose is to calculate paragraph ranges in real-time, right when the text storage is being edited. Knowing the correct paragraph ranges allows me to work with simple integers (as NSRange's) while calculating the selected rect. Otherwise I'd had to do a bunch of substrings to get to know whether the new line characters are selected or not. And those operations are really slow.
The implementation of my ParagraphTextStorage is here: https://github.com/CineDev/ParagraphTextKit
