Hi,

Issue of Segmentation fault (core dumped) for ue5-dev branch - 0.10.0

ue5-dev built from source on Ubuntu 22.04

Followed the final steps explained by @Morallez86.
Building was successful.
Ran two parallel manual_control.py which is updated with the safe distance sensor.
Sensor spawning and listening were successful.
When sensor is triggered, code crashes with Segmentation fault (core dumped).
We do not get expected print out from the Python side either (Vehicle too close: vehicle.type_id)
Here is the situation where sensor is triggered and debugging outputs.

As we see at above image, both vehicles were spawned and bounding boxes are visible, Terminal debugging prints the default overlap for each vehicle.

With this above image, blue vehicle's bounding box touches to the black's mesh and manual_control.py code crashed which spawned and manages the sensor for blue car with Segmentation fault (core dumped). From Terminal we see the debugging text that shows the overlap for that blue car has been detected (Overlaps found: 2) but not for the black car; so its debugging text remains 'Overlaps found:1'. Even though code crashes and pygame window closes, vehicle remains in the simulation.

This final image shows that the black vehicle moved to reverse and its bounding box touched to blue vehicle; code crashed with Segmentation fault (core dumped) and pygame closed, Overlap has been detected and can be seen from Terminal output text.

It seems we are getting memory issues due to UE5 manages it differently than UE4 while the tutorial is prepared for UE4 there might be some other reason as well. Any idea how to fix this issue, please?

UPDATE:

Same situation is true also for PostPhysTick which is been used by UE5 versions.

SOLUTION for custom sensor registry for ue5-dev

Actual credits go to tutorial owner @nsubiron
and the people who found the solutions for the crashes @roque2205, @jyb01124 and @Morallez86.

We implemented registering the same sensor to ue5-dev by using the same codes with corrections and with some necessary changes to align with UE5.

Thanks to CARLA community!

Here below are the codes with the relative paths

---

LibCarla/source/carla/sensor/data/SafeDistanceEvent.h

  // Copyright (c) 2019 Computer Vision Center (CVC) at the Universitat Autonoma
  // de Barcelona (UAB).
  //
  // This work is licensed under the terms of the MIT license.
  // For a copy, see <https://opensource.org/licenses/MIT>.
  
  #pragma once
  
  #include "carla/rpc/ActorId.h"
  #include "carla/sensor/data/Array.h"
  
  namespace carla {
  namespace sensor {
  namespace data {
  
    class SafeDistanceEvent : public Array<rpc::ActorId> {
    public:
  
      explicit SafeDistanceEvent(RawData &&data)
        // : Array<rpc::ActorId>(std::move(data)) {}
        : Array<rpc::ActorId>(0u, std::move(data)) {}
    };
  
  } // namespace data
  } // namespace sensor
  } // namespace carla

---

Unreal/CarlaUnreal/Plugins/Carla/Source/Carla/Sensor/SafeDistanceSensor.h

  // Copyright (c) 2019 Computer Vision Center (CVC) at the Universitat Autonoma
  // de Barcelona (UAB).
  //
  // This work is licensed under the terms of the MIT license.
  // For a copy, see <https://opensource.org/licenses/MIT>.
  
  #pragma once
  
  #include "Carla/Sensor/Sensor.h"
  #include "Carla/Actor/ActorDefinition.h"
  #include "Carla/Actor/ActorDescription.h"
  #include "Components/BoxComponent.h"
  
  #include "SafeDistanceSensor.generated.h"
  
  UCLASS()
  class CARLA_API ASafeDistanceSensor : public ASensor
  {
    GENERATED_BODY()
  
  public:  
  
    static FActorDefinition GetSensorDefinition();
  
    ASafeDistanceSensor(const FObjectInitializer &ObjectInitializer);
  
    void Set(const FActorDescription &ActorDescription) override;
  
    void SetOwner(AActor *Owner) override;
  
  //   void Tick(float DeltaSeconds) override;
    virtual void PostPhysTick(UWorld *World, ELevelTick TickType, float DeltaSeconds) override;
  
  
  private:
  
    UPROPERTY()
    UBoxComponent *Box = nullptr;
  };

---

Unreal/CarlaUnreal/Plugins/Carla/Source/Carla/Sensor/SafeDistanceSensor.cpp

  // Copyright (c) 2019 Computer Vision Center (CVC) at the Universitat Autonoma
  // de Barcelona (UAB).
  //
  // This work is licensed under the terms of the MIT license.
  // For a copy, see <https://opensource.org/licenses/MIT>.
  #include "Carla/Sensor/SafeDistanceSensor.h"
  #include "Carla.h"
  
  #include "Carla/Actor/ActorBlueprintFunctionLibrary.h"
  #include "Carla/Game/CarlaEpisode.h"
  #include "Carla/Util/BoundingBoxCalculator.h"
  #include "Carla/Vehicle/CarlaWheeledVehicle.h"
  
  ASafeDistanceSensor::ASafeDistanceSensor(const FObjectInitializer &ObjectInitializer)
    : Super(ObjectInitializer)
  {
    Box = CreateDefaultSubobject<UBoxComponent>(TEXT("BoxOverlap"));
    Box->SetupAttachment(RootComponent);
    Box->SetHiddenInGame(true); // Disable for debugging, should be true.
    Box->SetCollisionProfileName(FName("OverlapAll"));
  
    PrimaryActorTick.bCanEverTick = true;
  }
  
  FActorDefinition ASafeDistanceSensor::GetSensorDefinition()
  {
    auto Definition = UActorBlueprintFunctionLibrary::MakeGenericSensorDefinition(
        TEXT("other"),
        TEXT("safe_distance"));
  
    FActorVariation Front;
    Front.Id = TEXT("safe_distance_front");
    Front.Type = EActorAttributeType::Float;
    Front.RecommendedValues = { TEXT("1.0") };
    Front.bRestrictToRecommended = false;
  
    FActorVariation Back;
    Back.Id = TEXT("safe_distance_back");
    Back.Type = EActorAttributeType::Float;
    Back.RecommendedValues = { TEXT("0.5") };
    Back.bRestrictToRecommended = false;
  
    FActorVariation Lateral;
    Lateral.Id = TEXT("safe_distance_lateral");
    Lateral.Type = EActorAttributeType::Float;
    Lateral.RecommendedValues = { TEXT("0.5") };
    Lateral.bRestrictToRecommended = false;
  
    Definition.Variations.Append({ Front, Back, Lateral });
  
    return Definition;
  }
  
  void ASafeDistanceSensor::Set(const FActorDescription &Description)
  {
    Super::Set(Description);
  
    float Front = UActorBlueprintFunctionLibrary::RetrieveActorAttributeToFloat(
        "safe_distance_front",
        Description.Variations,
        1.0f);
    float Back = UActorBlueprintFunctionLibrary::RetrieveActorAttributeToFloat(
        "safe_distance_back",
        Description.Variations,
        0.5f);
    float Lateral = UActorBlueprintFunctionLibrary::RetrieveActorAttributeToFloat(
        "safe_distance_lateral",
        Description.Variations,
        0.5f);
  
    constexpr float M_TO_CM = 100.0f; // Unit conversion.
  
    float LocationX = M_TO_CM * (Front - Back) / 2.0f;
    float ExtentX = M_TO_CM * (Front + Back) / 2.0f;
    float ExtentY = M_TO_CM * Lateral;
  
    Box->SetRelativeLocation(FVector{LocationX, 0.0f, 0.0f});
  //   Box->SetBoxExtent(FVector{ExtentX, ExtentY, 0.0f});
    Box->SetBoxExtent(FVector{ExtentX, ExtentY, 100.0f});
  
  }
  
  void ASafeDistanceSensor::SetOwner(AActor *Owner)
  {
    Super::SetOwner(Owner);
  
    auto BoundingBox = UBoundingBoxCalculator::GetActorBoundingBox(Owner);
  
    Box->SetBoxExtent(BoundingBox.Extent + Box->GetUnscaledBoxExtent());
  }
  
  // void ASafeDistanceSensor::Tick(float DeltaSeconds)
  void ASafeDistanceSensor::PostPhysTick(UWorld *World, ELevelTick TickType, float DeltaSeconds)
  {
  //   Super::Tick(DeltaSeconds);
    TRACE_CPUPROFILER_EVENT_SCOPE(ASafeDistanceSensor::PostPhysTick);
    // safe check
    if (!IsStreamReady()) return;
  
    TSet<AActor *> DetectedActors;
    Box->GetOverlappingActors(DetectedActors, ACarlaWheeledVehicle::StaticClass());
    DetectedActors.Remove(GetOwner());
  
    if (DetectedActors.Num() > 0)
    {
      // UE_LOG(LogTemp, Warning, TEXT("[SAFE] DetectedActors=%d"), DetectedActors.Num());
      auto Stream = GetDataStream(*this);
      // Stream.Send(*this, GetEpisode(), DetectedActors);
      Stream.SerializeAndSend(*this, GetEpisode(), DetectedActors);
      // UE_LOG(LogTemp, Warning, TEXT("[SAFE] SerializeAndSend done"));
    }
  }

---

LibCarla/source/carla/sensor/s11n/SafeDistanceSerializer.h

  // Copyright (c) 2019 Computer Vision Center (CVC) at the Universitat Autonoma
  // de Barcelona (UAB).
  //
  // This work is licensed under the terms of the MIT license.
  // For a copy, see <https://opensource.org/licenses/MIT>.
  
  #pragma once
  
  #include "carla/Memory.h"
  #include "carla/MsgPack.h"
  #include "carla/rpc/ActorId.h"
  #include "carla/sensor/RawData.h"
  
  #include <cstdint>
  #include <cstring>
  
  namespace carla {
  namespace sensor {
  
    class SensorData;
  
  namespace s11n {
  
    class SafeDistanceSerializer {
    public:
  
      template <typename SensorT, typename EpisodeT, typename ActorListT>
      static Buffer Serialize(
          const SensorT &,
          const EpisodeT &episode,
          const ActorListT &detected_actors) {
        
        using ActorId = rpc::ActorId;
  
        std::vector<rpc::ActorId> ids;
        ids.reserve(detected_actors.Num());
  
        for (auto *actor : detected_actors) {
          if (auto ca = episode.FindCarlaActor(actor)) {
            ids.push_back(ca->GetActorId());
          }
        }
  
        const uint32_t count = static_cast<uint32_t>(ids.size());
        const size_t total_size = sizeof(uint32_t) + count * sizeof(ActorId);
  
        Buffer buffer(total_size);
        unsigned char *ptr = buffer.data();
  
        std::memcpy(ptr, &count, sizeof(uint32_t));
        ptr += sizeof(uint32_t);
  
        std::memcpy(ptr, ids.data(), count * sizeof(ActorId));
       
        return buffer;
      }
  
      static SharedPtr<SensorData> Deserialize(RawData &&data);
    };
  
  } // namespace s11n
  } // namespace sensor
  } // namespace carla

---

LibCarla/source/carla/sensor/s11n/SafeDistanceSerializer.cpp

  // Copyright (c) 2019 Computer Vision Center (CVC) at the Universitat Autonoma
  // de Barcelona (UAB).
  //
  // This work is licensed under the terms of the MIT license.
  // For a copy, see <https://opensource.org/licenses/MIT>.
  
  #include "carla/sensor/s11n/SafeDistanceSerializer.h"
  
  #include "carla/sensor/data/SafeDistanceEvent.h"
  
  namespace carla {
  namespace sensor {
  namespace s11n {
  
    SharedPtr<SensorData> SafeDistanceSerializer::Deserialize(RawData &&data) {
      return SharedPtr<SensorData>(new data::SafeDistanceEvent(std::move(data)));
    }
  
  } // namespace s11n
  } // namespace sensor
  } // namespace carla

---

Actual sensor registry

---

SensorRegistry.h at the following path has been modified:

Path: /CarlaUE5/LibCarla/source/carla/sensor/SensorRegistry.h

Add these to each relevant part in the same order:

  #include "carla/sensor/s11n/SafeDistanceSerializer.h" 
  
  class ASafeDistanceSensor; 
  
  std::pair<ASafeDistanceSensor *, s11n::SafeDistanceSerializer> 
  
  #include "Carla/Sensor/SafeDistanceSensor.h" 

---

The followings added to the PythonAPI.h

Path: /CarlaUE5/PythonAPI/carla/include/PythonAPI.h

Add:

    #include <carla/sensor/data/SafeDistanceEvent.h> 

---

Path:/CarlaUE5/PythonAPI/carla/src/SensorData.cpp

Add these to related parts, notice to quotes:
namespace data:

  namespace data { 
  
  '''  
  
  std::ostream &operator<<(std::ostream &out, const SafeDistanceEvent &meas) { 
  
  out << "SafeDistanceEvent(frame=" << std::to_string(meas.GetFrame()) 
  
  << ", timestamp=" << std::to_string(meas.GetTimestamp()) 
  
  << ", actor_count=" << std::to_string(meas.size()) 
  
  << ')'; 
  
  return out; 
  
  } 
  
  } 

void export_sensor_data:

  void export_sensor_data() {        
  
  ''' 
  
  class_<csd::SafeDistanceEvent, bases<cs::SensorData>, boost::noncopyable, std::shared_ptr<csd::SafeDistanceEvent>>("SafeDistanceEvent", no_init) 
  
  // size 
  
  .def("__len__", &csd::SafeDistanceEvent::size) 
  
  // iteration support 
  
  .def("__iter__", iterator<csd::SafeDistanceEvent>()) 
  
  // index access 
  
  .def("__getitem__", +[](const csd::SafeDistanceEvent &self, size_t pos) { 
  
  return self.at(pos); 
  
  }) 
  
  // string print 
  
  .def(self_ns::str(self_ns::self)) 
  
  ; 

---

Run the same commands in CarlaSetup.sh from CarlaUE5 path:

"Configuring the CARLA CMake project..."

  cmake -G Ninja -S . -B Build \ 
  
  --toolchain=$PWD/CMake/Toolchain.cmake \ 
  
  -DLAUNCH_ARGS="-prefernvidia" \ 
  
  -DCMAKE_BUILD_TYPE=Release \ 
  
  -DENABLE_ROS2=ON \ 
  
  -DPython_ROOT_DIR=${python_root} \ 
  
  -DPython3_ROOT_DIR=${python_root} \ 
  
  -DCARLA_UNREAL_ENGINE_PATH=$CARLA_UNREAL_ENGINE_PATH 

"Building CARLA..."

  cmake --build Build 

"Installing Python API..."

  cmake --build Build --target carla-python-api-install       

"Launching Carla - Unreal Editor..."

  cmake --build Build --target launch 

---

Test the Safe Sensor:

  bp = bp_lib.find('sensor.other.safe_distance')
  safe_sensor = world.spawn_actor(bp, carla.Transform(), attach_to=ego_vehicle)
  print("SAFE sensor spawned and listening...")
  
  # Use weakref for safety in callback
  world_ref = weakref.ref(world)
  
  def safe_callback(event):
      print("Actors inside safe distance:")
  
      for actor_id in event:
          # actor = world.get_actor(actor_id)
          actor = world_ref().get_actor(actor_id)
  
          if actor:
              print(" -", actor.type_id, "ID:", actor_id)
  
  # Register listener
  safe_sensor.listen(lambda data: safe_callback(data))

Expected output when other 2 vehicles are too close - tested from front side of the ego:

  Actors inside safe distance:
   - vehicle.dodge.charger ID: 70
   - vehicle.dodge.charger ID: 64
  Actors inside safe distance:
   - vehicle.dodge.charger ID: 70
   - vehicle.dodge.charger ID: 64
  Actors inside safe distance:
   - vehicle.dodge.charger ID: 70
   - vehicle.dodge.charger ID: 64
  Actors inside safe distance:
   - vehicle.dodge.charger ID: 70
   - vehicle.dodge.charger ID: 64