Your thought process of getting all the entities within a certain range of the player is a good start in my opinion! Below is an example of a method that uses the blocks in a player's line of sight (raytrace) to find the nearest entity that isn't obstructed.
public static Entity getNearestEntityInSight(Player player, int range) {
List<Entity> entities = player.getNearbyEntities(range, range, range); //Get the entities within range
Iterator<Entity> iterator = entities.iterator(); //Create an iterator
while (iterator.hasNext()) {
Entity next = iterator.next(); //Get the next entity in the iterator
if (!(next instanceof LivingEntity) || next == player) { //If the entity is not a living entity or the player itself, remove it from the list
iterator.remove();
}
}
List<Block> sight = player.getLineOfSight((Set) null, range); //Get the blocks in the player's line of sight (the Set is null to not ignore any blocks)
for (Block block : sight) { //For each block in the list
if (block.getType() != Material.AIR) { //If the block is not air -> obstruction reached, exit loop/seach
break;
}
Location low = block.getLocation(); //Lower corner of the block
Location high = low.clone().add(1, 1, 1); //Higher corner of the block
AxisAlignedBB blockBoundingBox = AxisAlignedBB.a(low.getX(), low.getY(), low.getZ(), high.getX(), high.getY(), high.getZ()); //The bounding or collision box of the block
for (Entity entity : entities) { //For every living entity in the player's range
//If the entity is truly close enough and the bounding box of the block (1x1x1 box) intersects with the entity's bounding box, return it
if (entity.getLocation().distance(player.getEyeLocation()) <= range && ((CraftEntity) entity).getHandle().getBoundingBox().b(blockBoundingBox)) {
return entity;
}
}
}
return null; //Return null/nothing if no entity was found
}
Note: There are a few things/potential bugs to consider when using the method above:
- The way the check whether the block's bounding box intersects with the entity's bounding box works means that the player might not actually be looking directly at the entity but only at part of the block. This proximity check can be done in a variety of different ways (for example: Checking whether the entity location is near the block's location) but most are not necessarily more precise. To get a more accurate result you'd need to make your own ray trace that scans smaller distances to make sure it doesn't "miss" any entities and to check whether the player is truly looking directly at the entity. The more precise you want it to be the more resources/time it will take. Check to see if this method is precise enough for you.
- The loop that looks through the nearby entities chooses the first one it finds that meets the criteria, meaning that if two or three entities are bunched up and close together, it might not choose the "true" closest one (this could be fixed with more checks though and might not be that important).
- The check whether an entity is obstructed only looks for blocks that are not air and therefore doesn't consider transparent blocks or blocks with small collision boxes (e.g. Looking through some stairs might return null even though you might technically be able to see the entity). Once again, this doesn't make a big difference usually and I think correcting it is rather tricky.