SPM for Modularized code
I think you're on the right track. Though I think it's simpler than you may think it is. There is no need to make frameworks to consume your swift packages before your app can consume them. I have a project with 12 local Swift Packages to modularize the code. No frameworks. Only 1 scheme. No hassle.
Baseline
Before I try to answer this question—since it's a bit involved—I want to start by laying a baseline that describes a very simple app with no dependencies for starters.
Imagine you just created a project and your file structure looks a bit like this:
RootDirectory/
MyProject.xcodeproj
MyProject/
MyProjectApp.swift
ContentView.swift
Info.plist
Assets.xcassets/
Create a Package
I generally stay away from creating my local swift packages within Xcode. I do it from the command line like so:
# Travel to the project's root directory
cd path/to/RootDirectory
# Create a directory to host the new package
mkdir MyPackage
# Move to the new directory
cd MyPackage
# Create the new Swift Package
swift package init
After that is done, the directory structure should look like this:
RootDirectory/
MyPackage/
Package.swift
Sources/
Tests/
MyProject.xcodeproj
MyProject/
MyProjectApp.swift
ContentView.swift
Info.plist
Assets.xcassets/
My Package.swift file looks like this:
// swift-tools-version:5.6
// The swift-tools-version declares the minimum version of Swift required to build this package.
import PackageDescription
let package = Package(
name: "Core",
platforms: [.iOS(.v15), .macOS(.v12), .watchOS(.v8), .tvOS(.v15)],
products: [
.library(
name: "Core",
targets: ["Core"]),
],
dependencies: [],
targets: [
.target(
name: "Core",
dependencies: []),
.testTarget(
name: "CoreTests",
dependencies: ["Core"]),
]
)
Consume the Local Package
After that is done, I would add the package to the App's Xcode Project like so:

After you've added the package, you want to make sure it shows up in the Linked Libraries area of the App Target's Build Phases:

Then, make sure your app target's scheme knows to include your packages for building and/or testing like so:

Result
You can do that over and over again. Your project should look something like this at the end:
