User Activity Logging System

A scalable, non-blocking user activity logging system for Spring Boot applications using PostgreSQL. This system provides selective method-level activity tracking through AOP (Aspect-Oriented Programming) with the @UserLogActivity annotation.

Architecture Overview
Features
Quick Start
Configuration
Usage Examples
Database Schema
API Endpoints
Monitoring
Performance Considerations
Troubleshooting

Architecture Overview

graph TB
    A[Business Method with @UserLogActivity] --> B[ActivityLoggingAspect]
    B --> C[ActivityEvent]
    C --> D[ActivityLogService]
    D --> E[In-Memory Queue]
    E --> F[Batch Processor]
    F --> G[UserActivityLogRepository]
    G --> H[PostgreSQL Database]
    
    I[Health Check] --> D
    J[REST API] --> G
    
    D --> K[Circuit Breaker]
    D --> L[Metrics]
    
    style G fill:#9c27b0,color:#ffffff
    style H fill:#1b5e20,color:#ffffff

Core Components

@UserLogActivity Annotation: Marks methods for activity logging
ActivityLoggingAspect: AOP aspect that intercepts annotated methods
ActivityEvent: Domain event containing activity information
ActivityLogService: Non-blocking service with batch processing using repository pattern
UserActivityLog Entity: JPA entity with optimized database indexes
UserActivityLogRepository: Spring Data JPA repository for database operations
Health Indicators: System monitoring and circuit breaker

Features

✅ Selective Logging: Only methods with @UserLogActivity annotation are logged
✅ Non-Blocking: Asynchronous processing doesn't impact business logic performance
✅ Scalable: Batch processing with configurable thresholds (100 events or 30 seconds)
✅ Resilient: Circuit breaker pattern prevents system overload
✅ Configurable: 11+ configuration options for fine-tuning
✅ Secure: Role-based access controls on query endpoints
✅ Observable: Health checks, metrics, and queue monitoring
✅ Production-Ready: Automatic cleanup, retention policies, error handling

Quick Start

1. Add the Annotation to Your Methods

@Service
public class ContributionService {
    
    @UserLogActivity(
        type = ActivityType.CONTRIBUTION_SUBMIT,
        message = "User #{#userId} submitted contribution: #{#arg0.title}"
    )
    public ContributionResult submitContribution(SubmitContributionCommand command) {
        // Your business logic here
        return new ContributionResult();
    }
}

2. Available Activity Types

public enum ActivityType {
    LOGIN,
    LOGOUT,
    AUTHENTICATION_FAILURE,
    API_CALL,
    DATA_CREATE,
    DATA_UPDATE,
    DATA_DELETE,
    FILE_UPLOAD,
    FILE_DOWNLOAD,
    PASSWORD_CHANGE,
    PROFILE_UPDATE,
    CONTRIBUTION_SUBMIT,
    WITHDRAWAL_SUBMIT,
    DOCUMENT_VIEW,
    REPORT_GENERATE,
    SYSTEM_ERROR
}

Configuration

Application Properties

# Activity Logging Configuration - Method-level logging only via @UserLogActivity annotation
app.activity-logging.enabled=true
app.activity-logging.batch-size=100
app.activity-logging.queue-capacity=10000
app.activity-logging.flush-interval=PT30S
app.activity-logging.retention-period=P90D
app.activity-logging.sampling-rate=1.0
app.activity-logging.circuit-breaker-failure-threshold=5
app.activity-logging.circuit-breaker-timeout=PT2M
app.activity-logging.include-request-payload=false
app.activity-logging.include-response-payload=false
app.activity-logging.max-payload-size=5000

Configuration Properties Explained

Property	Default	Description
`enabled`	`true`	Enable/disable the entire activity logging system
`batch-size`	`100`	Number of events to batch before database insert
`queue-capacity`	`10000`	Maximum in-memory queue size
`flush-interval`	`PT30S`	Maximum time to wait before flushing events
`retention-period`	`P90D`	How long to keep activity logs
`sampling-rate`	`1.0`	Percentage of events to log (0.0-1.0)
`circuit-breaker-failure-threshold`	`5`	Failures before circuit breaker opens
`circuit-breaker-timeout`	`PT2M`	Circuit breaker timeout duration
`include-request-payload`	`false`	Include HTTP request payloads (deprecated)
`include-response-payload`	`false`	Include HTTP response payloads (deprecated)
`max-payload-size`	`5000`	Maximum payload size in bytes (deprecated)

Usage Examples

Basic Method Logging

@UserLogActivity(type = ActivityType.API_CALL)
public String processData(String input) {
    // Method execution time and success/failure will be logged
    return "processed: " + input;
}

Enhanced Logging with SpEL Message Templates

@UserLogActivity(
    type = ActivityType.DATA_CREATE,
    message = "User #{#userId} created profile for #{#arg1} with email #{#arg2}"
)
public UserProfile createUserProfile(String userId, String name, String email) {
    return new UserProfile(userId, name, email);
}

Financial Operations Logging

@UserLogActivity(
    type = ActivityType.WITHDRAWAL_SUBMIT,
    message = "User #{#userId} submitted withdrawal of RM#{#arg1} to account #{#arg2}"
)
public WithdrawalResponse submitWithdrawal(String userId, Double amount, String bankAccount) {
    // Critical financial operations are automatically tracked
    return processWithdrawal(userId, amount, bankAccount);
}

File Operations Logging

@UserLogActivity(
    type = ActivityType.FILE_UPLOAD,
    message = "User #{#userId} uploaded document '#{#arg1}' (#{#arg2.length} bytes)"
)
public DocumentUploadResult uploadDocument(String userId, String fileName, byte[] fileContent) {
    return processFileUpload(userId, fileName, fileContent);
}

SpEL Expression Variables

The system supports Spring Expression Language (SpEL) in message templates with the following variables:

Variable	Description	Example
`#userId`	Current authenticated user ID	`#{#userId}`
`#ipAddress`	Client IP address	`#{#ipAddress}`
`#userAgent`	Client user agent	`#{#userAgent}`
`#timestamp`	Current timestamp	`#{#timestamp}`
`#args`	All method arguments as array	`#{#args[0]}`
`#arg0`, `#arg1`, etc.	Individual method arguments	`#{#arg0.name}`

Example with complex expressions:

@UserLogActivity(
    type = ActivityType.DATA_UPDATE,
    message = "User #{#userId} from #{#ipAddress} updated #{#arg0.getClass().getSimpleName()} with ID #{#arg0.id}"
)
public void updateEntity(BaseEntity entity) {
    // Business logic
}

Implementation Details

Repository Pattern Architecture

The system uses Spring Data JPA repository pattern for database operations:

// Entity Definition
@Entity
@Table(name = "app_user_activity_logs", indexes = {
    @Index(name = "idx_user_timestamp", columnList = "userId, timestamp"),
    @Index(name = "idx_timestamp", columnList = "timestamp"),
    @Index(name = "idx_activity_type", columnList = "activityType"),
    @Index(name = "idx_user_activity_type", columnList = "userId, activityType")
})
public class UserActivityLog {
    @Id
    @GeneratedValue(strategy = GenerationType.IDENTITY)
    private Long id;
    
    @Column(nullable = false, length = 100)
    private String userId;
    
    @CreationTimestamp
    @Column(nullable = false)
    private LocalDateTime timestamp;
    
    // ... other fields
}

// Repository Interface
@Repository
public interface UserActivityLogRepository extends JpaRepository<UserActivityLog, Long> {
    Page<UserActivityLog> findByUserIdOrderByTimestampDesc(String userId, Pageable pageable);
    
    @Modifying
    @Query("DELETE FROM UserActivityLog ual WHERE ual.timestamp < :cutoffTime")
    int deleteOldLogs(@Param("cutoffTime") LocalDateTime cutoffTime);
}

// Service Implementation
@Service
public class ActivityLogService {
    private void batchInsert(List<ActivityEvent> events) {
        List<UserActivityLog> activityLogs = events.stream()
                .map(this::convertToEntity)
                .toList();
        
        repository.saveAll(activityLogs);
    }
    
    private UserActivityLog convertToEntity(ActivityEvent event) {
        UserActivityLog log = new UserActivityLog();
        log.setUserId(event.getUserId());
        log.setTimestamp(event.getTimestamp());
        log.setActivityType(event.getActivityType());
        // ... map other fields
        return log;
    }
}

Benefits of Repository Pattern

Type Safety: No raw SQL strings, compile-time checking
Maintainability: Clean separation of concerns, easier to test
JPA Features: Automatic dirty checking, second-level caching, lazy loading
Query Methods: Spring Data JPA query derivation from method names
Transaction Management: Seamless integration with Spring's @Transactional
Performance: Optimized batch operations with saveAll()

Database Schema

UserActivityLog Table

CREATE TABLE app_user_activity_logs (
    id BIGSERIAL PRIMARY KEY,
    user_id VARCHAR(100) NOT NULL,
    timestamp TIMESTAMP NOT NULL DEFAULT CURRENT_TIMESTAMP,
    activity_type VARCHAR(50) NOT NULL,
    execution_time_ms BIGINT,
    ip_address VARCHAR(45),
    user_agent VARCHAR(1000),
    additional_context TEXT,
    correlation_id VARCHAR(100),
    success BOOLEAN,
    error_message TEXT
);

-- Optimized Indexes
CREATE INDEX idx_user_timestamp ON app_user_activity_logs (user_id, timestamp);
CREATE INDEX idx_timestamp ON app_user_activity_logs (timestamp);
CREATE INDEX idx_activity_type ON app_user_activity_logs (activity_type);
CREATE INDEX idx_user_activity_type ON app_user_activity_logs (user_id, activity_type);

API Endpoints

Get User Activities

GET /api/activity-logs/users/{userId}?page=0&size=20

Security: User can access own logs, admins can access any user's logs.

Get Activities by Type

GET /api/activity-logs/users/{userId}/types/CONTRIBUTION_SUBMIT?page=0&size=20

Get Activities in Date Range

GET /api/activity-logs/users/{userId}/range?startTime=2024-01-01T00:00:00&endTime=2024-12-31T23:59:59&page=0&size=20

Get System Statistics

GET /api/activity-logs/stats

Security: Admin only.

Response:

{
    "queueSize": 15,
    "droppedEventsCount": 0,
    "processedEventsCount": 1247,
    "circuitBreakerOpen": false
}

Monitoring

Health Check

The system provides a health indicator accessible via Spring Boot Actuator:

GET /actuator/health/activityLogging

Response:

{
    "status": "UP",
    "details": {
        "queueSize": 15,
        "queueCapacity": 10000,
        "queueUtilization": "0.15%",
        "processedEvents": 1247,
        "droppedEvents": 0,
        "circuitBreakerOpen": false,
        "batchSize": 100,
        "flushIntervalSeconds": 30
    }
}

Circuit Breaker States

State	Description	Action
CLOSED	Normal operation	Events are processed normally
OPEN	System overloaded	Events are dropped, circuit breaker timeout active
HALF-OPEN	Recovery testing	Limited events processed to test system recovery

Performance Considerations

Optimal Configuration

For high-volume applications (>1000 events/minute):

app.activity-logging.batch-size=100
app.activity-logging.flush-interval=PT30S
app.activity-logging.queue-capacity=10000

For low-volume applications (<100 events/minute):

app.activity-logging.batch-size=25
app.activity-logging.flush-interval=PT10S
app.activity-logging.queue-capacity=2000

Memory Usage

Each event consumes approximately 500 bytes to 2KB of memory
With default settings (10,000 queue capacity), maximum memory usage is ~20MB
Adjust queue-capacity based on available memory and expected load
Repository pattern reduces memory overhead compared to raw JDBC operations

Database Performance

Repository Pattern: Uses Spring Data JPA for optimized database operations
Batch Processing: repository.saveAll() provides efficient bulk insert operations
JPA Entity Mapping: Automatic conversion from ActivityEvent to UserActivityLog entity
Optimized Indexes: Database indexes are optimized for common query patterns
Entity Relationships: Leverages JPA benefits like dirty checking and caching
Consider partitioning app_user_activity_logs table for very high volumes (>1M records/month)

Troubleshooting

Common Issues

Events Not Being Logged

Check if logging is enabled:
```
app.activity-logging.enabled=true
```

Verify annotation placement:

// ✅ Correct - on public method in Spring-managed bean
@Service
public class MyService {
    @UserLogActivity(type = ActivityType.API_CALL)
    public void myMethod() { ... }
}

// ❌ Incorrect - on private method
@UserLogActivity(type = ActivityType.API_CALL)
private void myMethod() { ... }

Check circuit breaker status:
```
GET /actuator/health/activityLogging
```

High Memory Usage

Reduce queue capacity:

app.activity-logging.queue-capacity=2000

Increase batch size for faster processing:
```
app.activity-logging.batch-size=100
```

Use simple message templates:

@UserLogActivity(
    type = ActivityType.API_CALL,
    message = "Basic API call logged"
)

Database Performance Issues

Check database indexes:

EXPLAIN ANALYZE SELECT * FROM app_user_activity_logs 
WHERE user_id = 'user123' ORDER BY timestamp DESC LIMIT 20;

Consider table partitioning:

-- Partition by month for high-volume systems
CREATE TABLE app_user_activity_logs_2024_01 
PARTITION OF app_user_activity_logs 
FOR VALUES FROM ('2024-01-01') TO ('2024-02-01');

Clean up old logs regularly: The system automatically cleans up logs older than the retention period, but you can also manually clean up:
```
DELETE FROM app_user_activity_logs 
WHERE timestamp < NOW() - INTERVAL '90 days';
```

Debug Logging

Enable debug logging to troubleshoot issues:

logging.level.com.isianpadu.mywira.module.activitylog=DEBUG

Monitoring Queries

Check queue backlog:

SELECT 
    COUNT(*) as total_logs,
    MAX(timestamp) as latest_log,
    MIN(timestamp) as oldest_log
FROM app_user_activity_logs 
WHERE timestamp > NOW() - INTERVAL '1 hour';

Check most active users:

SELECT 
    user_id,
    COUNT(*) as activity_count,
    MAX(timestamp) as last_activity
FROM app_user_activity_logs 
WHERE timestamp > NOW() - INTERVAL '24 hours'
GROUP BY user_id 
ORDER BY activity_count DESC 
LIMIT 10;

Support

For questions or issues:

Check the troubleshooting section
Review application logs with debug logging enabled
Monitor health check endpoint for system status
Create an issue in the project repository

License

This project is part of MyWira Backend system. See project license for details.

alifhaikal88/MWP_USER_ACTIVITY_LOGGING.md