I thankfully managed to find an answer to my question. The solution involved creating a custom layout and referencing that in my preferences.xml file. Here's how it broke down:
preferences.xml
<?xml version="1.0" encoding="utf-8"?>
<PreferenceScreen xmlns:android="http://schemas.android.com/apk/res/android"
android:fragment="com.app_name.fragment">
<EditTextPreference
android:id="@+id/key_Edit"
android:clickable="true"
android:inputType="textCapCharacters|textMultiLine"
android:enabled="true"
android:gravity="top"
android:key="pref_phrase"
android:singleLine="true"
android:textAlignment="center"
android:title="@string/preference_key"
android:defaultValue=" "
android:selectAllOnFocus="true"
android:fontFamily="@font/my_font"
android:textColor="@color/colorPrimary"
android:layout="@layout/preference_layout"/>
<EditTextPreference
android:id="@+id/password_Edit"
android:gravity="top"
android:key="pref_password"
android:selectAllOnFocus="true"
android:singleLine="true"
android:textAlignment="center"
android:title="@string/preference_security"
android:defaultValue=" "
android:inputType="number"
android:maxLength="5"
android:fontFamily="@font/my_font"
android:textColor="@color/colorPrimary"
android:layout="@layout/preference_layout"/>
</PreferenceScreen>
preference_layout.xml
<?xml version="1.0" encoding="utf-8"?>
<android.support.constraint.ConstraintLayout
xmlns:android="http://schemas.android.com/apk/res/android"
xmlns:app="http://schemas.android.com/apk/res-auto"
xmlns:tools="http://schemas.android.com/tools"
android:layout_width="match_parent"
android:layout_height="?attr/listPreferredItemHeight"
android:paddingEnd="?attr/listPreferredItemPaddingRight"
android:paddingLeft="?attr/listPreferredItemPaddingLeft"
android:paddingRight="?attr/listPreferredItemPaddingRight"
android:paddingStart="?attr/listPreferredItemPaddingLeft">
<android.support.constraint.ConstraintLayout
android:id="@+id/layout"
android:layout_width="match_parent"
android:layout_height="wrap_content"
app:layout_constraintBottom_toBottomOf="parent"
app:layout_constraintEnd_toEndOf="parent"
app:layout_constraintHorizontal_bias="0.5"
app:layout_constraintStart_toStartOf="parent"
app:layout_constraintTop_toTopOf="parent">
<TextView
android:id="@android:id/title"
android:layout_width="match_parent"
android:layout_height="wrap_content"
android:fontFamily="@font/my_font"
android:textAppearance="@style/TextAppearance.AppCompat.Medium"
android:textColor="@color/colorPrimary"
app:layout_constraintEnd_toEndOf="parent"
app:layout_constraintHorizontal_bias="0.5"
app:layout_constraintStart_toStartOf="parent"
app:layout_constraintTop_toTopOf="@id/layout" />
<TextView
android:id="@android:id/summary"
android:layout_width="match_parent"
android:layout_height="wrap_content"
android:fontFamily="@font/my_font"
android:textAppearance="@style/TextAppearance.AppCompat.Small"
android:textColor="@color/colorPrimary"
app:layout_constraintEnd_toEndOf="parent"
app:layout_constraintHorizontal_bias="0.5"
app:layout_constraintStart_toStartOf="parent"
app:layout_constraintTop_toBottomOf="@android:id/title"
android:paddingTop="4dp"/>
</android.support.constraint.ConstraintLayout>
</android.support.constraint.ConstraintLayout>
If you intend to also modify the font of the dialog that inflates with these preferences, you will have to create a custom EditTextPreference and work with the associated dialog builder. Hope it helps you!
EDIT
If you are comfortable using styles.xml, EditTextPreferences ultimately inherit from AlertDialogs in creating and displaying their dialogs. Thus, you can create a custom theme for AlertDialogs in which you set the fontFamily. Thus:
<style name="AlertDialogTheme" parent="Theme.AppCompat.Light.Dialog.Alert">
<item name="android:fontFamily">@font/my_font</item>
<item
name="android:textAppearance">@style/TextAppearance.AppCompat.Medium</item>
<item name="android:textStyle">bold</item>
<item name="android:windowNoTitle">true</item>
</style>
Lastly, override the standard AlertDialogTheme in your application's theme using that custom AlertDialogTheme.